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Prednisolone Versus Dexamethasone 
for Croup: a Randomized 
Controlled Trial 


Colin M. Parker, MBChB, DCH, MRCPCH, FACEM,*” Matthew N. Cooper, BCA, BSc, PhD° 


OBJECTIVES: The use of either prednisolone or low-dose dexamethasone in the treatment of 
childhood croup lacks a rigorous evidence base despite widespread use. In this study, we 
compare dexamethasone at 0.6 mg/kg with both low-dose dexamethasone at 0.15 mg/kg and 
prednisolone at 1 mg/kg. 


METHODS: Prospective, double-blind, noninferiority randomized controlled trial based in 1 
tertiary pediatric emergency department and 1 urban district emergency department in Perth, 
Western Australia. Inclusions were age >6 months, maximum weight 20 kg, contactable by 
telephone, and English-speaking caregivers. Exclusion criteria were known prednisolone or 
dexamethasone allergy, immunosuppressive disease or treatment, steroid therapy or 
enrollment in the study within the previous 14 days, and a high clinical suspicion of an 
alternative diagnosis. A total of 1252 participants were enrolled and randomly assigned to 
receive dexamethasone (0.6 mg/kg; n = 410), low-dose dexamethasone (0.15 mg/kg; n = 410), 
or prednisolone (1 mg/kg; n = 411). Primary outcome measures included Westley Croup 
Score 1-hour after treatment and unscheduled medical re-attendance during the 7 days after 
treatment. 


RESULTS: Mean Westley Croup Score at baseline was 1.4 for dexamethasone, 1.5 for low-dose 
dexamethasone, and 1.5 for prednisolone. Adjusted difference in scores at 1 hour, compared 
with dexamethasone, was 0.03 (95% confidence interval —0.09 to 0.15) for low-dose 
dexamethasone and 0.05 (95% confidence interval —0.07 to 0.17) for prednisolone. Re- 
attendance rates were 17.8% for dexamethasone, 19.5% for low-dose dexamethasone, and 
21.7% for prednisolone (not significant [P = .59 and .19]). 


CONCLUSIONS: Noninferiority was demonstrated for both low-dose dexamethasone and 
prednisolone. The type of oral steroid seems to have no clinically significant impact on 
efficacy, both acutely and during the week after treatment. 


QE 


WHAT’S KNOWN ON THIS SUBJECT: Although dexamethasone at 0.6 mg/kg is 
an established evidence-based treatment of childhood croup (reducing 
hospital admissions, length of stay, and need for endotracheal intubation), 
alternative corticosteroid regimes are in widespread use based on evidence 
from small studies and observational data. 
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ARTICLE 


The efficacy and safety of 
corticosteroids for the treatment of 
croup has established their use as 
routine therapy for the emergency 
department (ED) management of 
croup, > including, at our own 
institution, all ED attendances for 
croup.” Steroid treatments have been 
shown to significantly decrease the rate 
of hospital admission, length of hospital 
stay, return visits, endotracheal 
intubation, and admission to ICUs in 
patients with croup.” ” Although 
different routes of corticosteroid 
administration have been used, 
including nebulized, intramuscular, and 
intravenous dosing, the oral route has 
many advantages and is the preferred 
route in many centers.” The 
clinician treating children with croup 
may further consider the type and dose 
of oral steroid. 


Although early trials revealed safety 
and efficacy with intramuscular 
dexamethasone at a dose of 

0.6 mg/kg,° subsequent studies 
revealed efficacy of oral 
dexamethasone,’ and there is evidence 
that smaller doses of dexamethasone 
(0.3 and 0.15 mg/kg po) may be 
equally efficacious.’* The lower oral 
dose of 0.15 mg/kg has been accepted 
and implemented in some centers’; 
however, the limited number of 
supporting studies’*“ raises the 
question of efficacy compared with 

a dose of 0.6 mg/kg po,””” particularly 
around the power of these studies to 
detect small, but clinically important, 
differences in outcomes. Notably, the 
studies by Geelhoed and Macdonald!’ 
had only 80% power (at the 5% 
significance level) to detect a doubling 
in the duration of hospitalization, 
which would only reveal a relatively 
large clinical difference between 
treatments. On the other hand, the 
lower dose is supported by almost 3 
decades of clinical experience at our 
institution, indicating no rise in the rate 
of intubation or admission to intensive 
care after a reduction of the dose to 
0.15 mg/kg po.’ The well-proven 
efficacy of prednisolone in a dose of 


1 mg/kg po for croup in patients 
receiving intubation suggests that 

a comparable glucocorticoid dose of 
dexamethasone (~0.15 mg/kg) should 
be equally effective.’ 


Prednisolone (1 mg/kg) has been 
shown in 1 study to shorten the time 
to extubation for patients with croup 
in intensive care.“ Many centers 
therefore use oral prednisolone for 
the ED management of croup as 

a more readily available alternative to 
dexamethasone. ° The type and dose 
of corticosteroid treatment of croup 
depends largely on geographic 
location because different centers 
preferentially administer different 
drugs and doses. There have only 
been 3 small randomized controlled 
trials used to compare oral 
dexamethasone with oral 
prednisolone. Fifoot and Ting’? found 
no difference between the 2 
treatments, and an earlier study by 
Sparrow and Geelhoed?” of 133 
children with mild to moderate croup 
revealed that children treated with 
prednisolone (1 mg/kg po) were 
more likely to seek unscheduled 
follow-up for medical care than 
children treated with dexamethasone 
(0.15 mg/kg po). Garbutt et alë 
found no difference in duration of 
croup symptoms or additional health 
care when comparing a single dose of 
dexamethasone with 3 days of 
prednisolone treatment. 


In this study, we aim to compare the 
traditional, evidence-supported gold 
standard croup treatment, 
dexamethasone at a dose of 

0.6 mg/kg, with 2 alternate 
treatments already in widespread 
use, namely lower-dose 
dexamethasone (0.15 mg/kg) and 
prednisolone (1 mg/kg), and assess 
these treatments for noninferiority. 


METHODS 


Design 


We conducted a prospective, double- 
blind, randomized controlled trial at 2 


urban EDs: Princess Margaret 
Hospital for Children (a tertiary 
pediatric center) and Joondalup 
Health Campus (an urban district 
hospital). Hospital research ethics 
committee approval was obtained at 
both centers, and the study was 
registered with the Australian New 
Zealand Clinical Trials Registry 
(identifier ACTRN12609000290291). 
De-identified individual participant 
data will not be made available. 


Participants 


Patients were a convenience sample 
of children presenting to either of the 
EDs with croup during the study 
period (March 2009-July 2012). The 
disease state was defined as a clinical 
diagnosis of croup (laryngotracheitis) 
by the ED doctor after a history and 
physical examination. If there was any 
diagnostic uncertainty, clinicians 
could refer to a Guidance for Doctors 
information sheet (Supplemental 
Information), in which a croup 
diagnosis is defined as a hoarse voice 
or barking cough and stridor (with or 
without increased work of breathing) 
directly observed or elicited in the 
history. Inclusion criteria were age 
>6 months, contactable by telephone, 
and English-speaking caregivers. A 
maximum weight of 20 kg was 
imposed to limit the maximum 
possible dexamethasone dose to 

12 mg (adult dose). Exclusion criteria 
included known prednisolone or 
dexamethasone allergy, 
immunosuppressive disease or 
treatment, steroid therapy or 
enrollment in the same study within 
the previous 14 days, and a high 
clinical suspicion of an alternative 
diagnosis, with specific prompts to 
include bacterial tracheitis, inhaled 
foreign body, retropharyngeal 
abscess, epiglottitis, angioedema, 
vascular ring, and subglottic stenosis. 
Signed consent was obtained from 
caregivers by the treating doctor after 
the doctor provided a standardized 
information sheet (Croup Study: 
Information for Parents, see 
Supplemental Information). 
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Interventions 


Patients were randomly assigned to 1 
of 3 interventions: dexamethasone 
(0.6 mg/kg; standard “control” 
treatment), low-dose dexamethasone 
(0.15 mg/kg), or prednisolone 

(1 mg/kg). The randomization list 
was computer-generated at www. 
randomization.com by using 
randomly permuted block sizes in the 
ratio of 4 patients from each group, 
with block randomization by center. 
All medications were prepared, 
randomly assigned, and labeled by 

a Clinical trials pharmacist at Princess 
Margaret Hospital. To preserve the 
influence of palatability, no masking 
agents were used; however, 
individual doses of the trial 
medication were identically 
packaged, dispensed via a dispensing 
rack in order of randomization, and 
dosed at 0.3 mL/kg by the nurse after 
enrollment and initial severity 
scoring. Both staff (administering and 
assessing treatments) and patients 
were therefore blinded to the 
treatment allocation. 


Outcomes 


The study had 2 primary outcome 
measures: (1) an objective and 
validated**” measure of croup 
severity, the Westley Croup Score 
(WCS),*° and (2) re-attendance for 
follow-up of ongoing symptoms. 


The WCS is a clinical croup score, 
with a range of 0 to 17 points, that is 
based on stridor, retractions, air entry, 
cyanosis, and level of consciousness 
(Supplemental Fig 5). It is possible to 
have a clinical diagnosis of croup that 
is based on a history of a barking 
cough and yet have a WCS of 0, with 
no stridor or retractions and normal 
air entry. The WCS was assessed at 
baseline, at 1 hour after treatment, 
hourly up to 6 hours, and again at 
12 hours for patients not yet 
discharged from the hospital; patients 
were discharged when clinically 
appropriate (Croup Oral Steroid 
Study: Guidance for Doctors, see 
Supplemental Information). The 


second primary outcome was 
unscheduled medical re-attendance 
(for any reason) during the 7 days 
after treatment. This information was 
obtained via a telephone call to 
caregivers within 4 weeks after 
discharge. If the caregiver was 
deemed as not contactable (after 
several failed contact attempts), re- 
attendance to the ED with a diagnosis 
of croup was searched for 
electronically. 


Secondary outcomes included total 
hospital stay, ED length of stay, 
vomiting, use of nebulized 
epinephrine (adrenaline), 
endotracheal intubation, need for 
additional steroid doses, and need for 
admission to an inpatient ward, 
emergency short-stay unit, or ICU. 


Statistical Analysis 


The study was designed to 
demonstrate the noninferiority** of 
low-dose dexamethasone and 
prednisolone relative to the standard 
full dose of dexamethasone for the 2 
primary outcomes: (1) WCS (from 
baseline) at 1 hour and subsequent 
hourly intervals after administration 
and (2) the rate of unscheduled 
medical re-attendance in the 7 days 
after administration. The analysis was 
conducted per protocol via intention 
to treat; noninferiority was specified 
as the upper bound of the 2-sided 
95% confidence interval (CI) for the 
reduction in WCS (for the 
intervention group [low-dose 
dexamethasone or prednisolone] 
relative to the standard treatment 
group [dexamethasone]) not 
exceeding 0.5. Whereas authors of 
some previous studies have used 

a WCS of 1 as a clinically important 
difference,” we selected 0.5 to 
increase the study’s discriminatory 
ability” among milder croup cases 
(scores between 0 and 2), which 
typically account for the majority of 
patients.” Under this design, it was 
determined that 437 participants 
would be required, per arm, to 
provide 90% power to demonstrate 


noninferiority (by using a 2-sided 

t test; common SD of 1.8; a = .05). 
The expected re-attendance rate for 
croup at our institution is 4% to 6%’; 
this sample size would provide 85% 
power to detect an absolute 
difference in re-attendance rate of 5% 
(based on 10% vs 5%; a = .05). 


Descriptive statistics are presented 
as mean and SD, median, or count 
and percentage, as appropriate. A 
per-protocol analysis was used 

to employ linear regression, 

adjusted for baseline levels, of 

the postadministration hourly 
assessment of the WCS to calculate 
95% CIs for the difference between 
groups. The x^ test was used to 
compare the frequency of 
dichotomous and categorical 
variables; a 1-way analysis of 
variance or Student's t test was used 
to compare parametric variables, 
either between all groups or between 
each intervention group and the 
control group, respectively. Secondary 
outcomes were analyzed by using the 
same techniques; because of 

a moderate right skew, length of stay 
was analyzed after a log 
transformation, and the ratio of 
geometric means between groups is 
reported. For patients whose WCS 
was not recorded at an hourly 
assessment, if, and only if, their 
baseline WCS was 0 and they were 
discharged within the 60 minutes 
preceding that assessment, their 
score was assumed to be 0 for that 
assessment. Post hoc consistency 
analyses included additional 
modeling approaches to validate the 
per-protocol analysis. This included 
dichotomizing patients according to 
their status of recovered (either 
discharged or with a WCS of 0 at the 
hourly clinical review [assuming it 
was not 0 at baseline]) versus not 
recovered (still within the hospital) 
or improved (either discharged or 
with a WCS lower at the hourly 
clinical review than at baseline) 
versus not improved (still within the 
hospital) and using logistic regression 
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to generate odds ratios (ORs) and 
95% CIs for each treatment relative 
to the control group. The WCS at the 
hourly clinical review was also 
analyzed unmodified via ordinal 
logistic regression to generate ORs 
with 95% CIs. Models were adjusted 
for age and study center and, when 
appropriate, baseline WCS. Because of 
the higher than anticipated number of 
patients with multiple attendances 
within the study, post hoc analyses 
were conducted to calculate the 
intraclass correlation coefficient and 
to replicate the per-protocol analysis 
with clustering on subject 
identification. All data manipulation 
and analysis were conducted in R.” 


Patient and Public Involvement 


There was no patient and public 
involvement in the study design or 
implementation. No patients or their 
representatives were asked to help 
interpret or disseminate the results. 


RESULTS 


Trial Population 


A total of 1252 patient attendances 
underwent random assignment into 
the trial. After exclusions (Fig 1), 
1231 patients entered the analysis 
set; 410 were assigned to 
dexamethasone (0.6 mg/kg), 410 
were assigned to low-dose 
dexamethasone (0.15 mg/kg), and 
411 assigned to prednisolone 

(1 mg/kg). These 1231 attendances 
included 105 repeat enrollments: 48 
patients enrolling twice and a further 
3 patients enrolling 3 times each, all 
outside the 14-day exclusion period. 
Twenty-eight patients were enrolled 
despite meeting =1 exclusion 
criterion; 19 children weighing >20 
kg, 4 children with laryngomalacia, 4 
children with steroid use in the 14 
preceding days, and 3 children 

<6 months of age all were included in 
the analysis on an intention-to-treat 
basis. The distribution of repeat 
enrollments and patients meeting an 
exclusion criterion was relatively 


balanced across treatment groups 
(dexamethasone/low-dose 
dexamethasone/prednisolone 
distributed at 1:1.48:1.14 and 1:1.29: 
1.71, respectively). The mean age and 
weight of patients was 30 months and 
14.0 kg, respectively; 38% of patients 
were girls, and these characteristics, 
in addition to croup severity at 
presentation (baseline WCS), did not 
differ between groups (Table 1). 


Primary Outcomes 


The percentage of patients available 
for the 1-hour assessment was 
similar between groups at 88.3%, 
88.3%, and 89.1% for 
dexamethasone, low-dose 
dexamethasone, and prednisolone, 
respectively. There was no 
Statistically significant difference 
between the 3 groups for the WCS at 
the 1-hour assessment, with the 
adjusted difference in scores at 

1 hour (relative to the 
dexamethasone group) being 0.03 
(95% CI —0.09 to 0.15) for low-dose 
dexamethasone and 0.05 (95% CI 
—0.07 to 0.17) for prednisolone 
(Table 2, Fig 2). The upper limits of 
these CIs fall within the prespecified 
noninferiority margin of 0.5. 


Re-attendance rates (Table 2) were 
modest at 17.8% (dexamethasone), 
19.5% (low-dose dexamethasone), 
and 21.7% (prednisolone), and 
similarly, ED re-attendance rates were 
low at 5.9% (dexamethasone), 8.8% 
(low-dose dexamethasone), and 7.5% 
(prednisolone), with no statistical 
difference between treatment groups 
(Table 3). 


For the low-dose dexamethasone 
group relative to the dexamethasone 
group, the WCS was 0.11 higher at 

2 hours and 0.23 higher at 3 hours 
(Fig 2); although the difference was 
significant at the 3-hour assessment 
(P = .04), the upper limit of the 

95% CI (0.45) was within the 
noninferiority margin (Table 4, Fig 3). 
These estimates were 0.04 at 2 hours 
and 0.04 at 3 hours for the 
prednisolone group, with the upper 


limit of the 95% CIs not crossing 0.5. 
These results must be interpreted 
with caution given the lower 
numbers. 


Consistency analyses for the odds of 
recovery (WCS of 0 or discharged) 
and improvement (lower WCS) are 
shown in Table 4, with no statistically 
significant differences demonstrated. 


Secondary Outcomes and Adverse 
Events 


The median length of stay across 
patients was 124 minutes; this did 
not differ significantly between the 
study groups (P = .23), as reflected in 
the number of patients for which 
hourly assessment data were 
available (87.4%, 12.4%, 4.2% for 
hours 1, 2, and 3, respectively). The 
ratio of geometric means for total 
length of stay, relative to 
dexamethasone, was 0.99 (95% CI 
0.92 to 1.07) and 1.04 (95% CI 0.97 
to 1.12) for low-dose dexamethasone 
and prednisolone, respectively 

(Fig 4). There was no difference (P = 
.63) between groups in the 
percentage of participants whose 
total length of stay exceeded 4 hours 
(8.8%, 7.1%, and 8.5% for 
dexamethasone, low-dose 
dexamethasone, and prednisolone, 
respectively). The intraclass 
correlation coefficient was lower for 
the recovery at 1 hour (0.17) but 
higher for length of stay (0.52); 
despite this, incorporation of 
clustering only saw changes to the 
SEs in the third and fourth decimal 
place; therefore, per-protocol 
analyses are presented. 


There were no differences between 
treatment groups in the need for 
nebulized epinephrine (2.2%-3%) or 
in incidence of vomiting (up to 4%) 
after treatment (Table 3). A repeat 
dose of epinephrine was given to 0%, 
1.2%, and 1.0% of participants in the 
dexamethasone, low-dose 
dexamethasone, and prednisolone 
groups, respectively. No study 
participants required intubation, and 
none were admitted to intensive care 
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Enrollment 


Eligible patients during study 
period (N = 8707) 


Not assessed or excluded 
(including refusal) 
(n = 7476) 


Randomly assigned (n = 1231) 


Allocation Allocated to Allocated to low-dose Allocated to 
dexamethasone at 0.6 mg/kg dexamethasone at 0.15 mg/kg prednisolone at 1 mg/kg 
(n = 410) (n = 410) (n = 411) 
»Received allocated intervention »Received allocated intervention »Received allocated intervention 
(n = 410) (n = 410) (n = 411) 
»Did not receive allocated »Did not receive allocated »Did not receive allocated 
intervention (n = 0) intervention (n = 0) intervention (n = 0) 
Follow-up Telephone follow-up (n = 286) Telephone follow-up (n = 286) Telephone follow-up (n = 282) 
ED records follow-up (n = 124) ED records follow-up (n = 124) ED records follow-up (n = 129) 
Discontinued intervention (n = 0) Discontinued intervention (n = 0) Discontinued intervention (n = 0) 
Analysis Analyzed (n = 410) Analyzed (n = 410) Analyzed (n = 411) 
Excluded from analysis (n = 0) Excluded from analysis (n = 0) Excluded from analysis (n = 0) 
FIGURE 1 


Consolidated Standards of Reporting Trials diagram of flow of participants through trial stages. 


during their hospitalization. One or 
more additional steroid doses were 
given to 11.3%, 15.1%, and 18.9% of 
participants in the dexamethasone, 
low-dose dexamethasone, and 
prednisolone groups, respectively 

(P = .04). 


Adverse events were reported in only 4 
patients; 1 child assigned to 
dexamethasone had a 30-second febrile 


convulsion ~30 minutes after dosing, 
which was not attributed to the 
medication by the treating clinicians; 1 
child assigned to prednisolone 
developed insomnia (dose at ~5:00 pm 
and awake until 3:00 am); 1 child 
assigned to low-dose dexamethasone 
was transferred back to the ED from the 
ED short-stay unit and treated with 
nebulized epinephrine for possible 
stridor; and 1 patient assigned to 


low-dose dexamethasone developed 
hyperactivity 30 minutes after the dose. 


DISCUSSION 


The ToPDoG (Trial of Prednisolone/ 
Dexamethasone Oral Glucocorticoid) 
study, is, to the best of our 
knowledge, the largest croup 
randomized controlled trial published 
to date. Our findings confirm the 
clinical experience of safety** and 


TABLE 1 Baseline Patient Characteristics, by Treatment Group efficacy**?° of oral steroids for 
Variable Dexamethasone Low-Dose Prednisolone croup. We studied 2 different but 
(Standard Treatment) Dexamethasone complementary primary outcome 
i 410 410 411 measures: an objective measure of 
Demographic variables acute severity and improvement (the 
Age at presentation, mean (SD), mo 29.2 (17.3) 30.5 (16.3) 30.4 (16.2) WCS) and also a real-world, clinically 
Female sex, n (%) 160 (39.0) 156 (38.0) 152 (37.0) relevant outcome, re-attendance rate, 
Wt, mean (SD), kg 13.8 (3.7) 14.1 (3.4) 14.0 (3.8) which has implications for patient 
Baseline characteristics 
WCS at enrollment, mean (SD) 1.4 (1.4) 1.5 (1.4) 1.5 (1.4) and family satisfaction as well as use 
WCS category at enrollment, n (%) of resources in hospitals and the 
0-1 226 (55.4) 216 (52.9) 224 (54.5) wider community. 
2—ő 197 (58.5) 165 (40.4) 160 (38.9) 
4+ 25 (6.1) 27 (6.6) 27 (6.6) Dexamethasone is generally not 


P value were calculated by using Student’s t test for continuous outcomes and the x test for categorical outcomes. available outside of the hospital 
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TABLE 2 WCS and Re-attendance, by Treatment Group 


Variable Dexamethasone (Standard Low-Dose 8 Coefficient (95% 
Treatment) Dexamethasone Cl) 
WCS at 1 h 0.45 (0.92) (n = 562) 0.48 (0.95) (n = 562)  .035 (—0.09 to 0.15) 


Change in WCS to 1 h 


Re-attendance rate (GP 
and ED) 


1.05 (1.25) (n = 562) 


73 (17.8%) 


1.07 (1.25) (n = 562) = 


80 (19.5%) = 


P Prednisolone B Coefficient (95% P 


Cl) 


62 0.49 (1.04) (n= 0 (—0.07 to 0.17) .40 
366) 

— 103: (1.17) (n= — = 
366) 


09 89 (21.7%) = 19 


Data are presented as mean (SD) or count (%), as appropriate. B coefficients (95% Cl) represent the differences between the treatment groups and standard treatment (dexamethasone) 
in the WCS at the follow-up assessment and was calculated by using linear regression adjusted for age, baseline WCS, and study center. The P value for difference was calculated by using 


linear regression or the x? test. —, not applicable. 


environment, so there is a distinct 
advantage of being able to use 
prednisolone to treat croup in the 
community setting. We chose not to 
use masking agents in the 
preparation of trial medications 
because palatability issues affect the 
real-world utility of these 
medications, especially in pediatric 
populations. 


A number of limitations were 
applicable to our study. The study 


WCS 


Baseline 1 Hour 


population was a convenience sample 
from 2 institutions, and ~1 in 7 
patients with croup were enrolled. 
Our power calculation (based on 
hypothesis testing) predicted that 
1311 patients were required; we only 
enrolled 1252 subjects. However, the 
CIs in our data suggest that our 
sample was large enough to answer 
the clinical questions posed. 


We were unable to record the 
number of participants screened for 


Treatment 


=®- Dexamethasone (standard) 


=*- Low-dose dexamethasone 


"E + Prednisolone 


2 Hours 


Assessment 


FIGURE 2 


Westley Croup Score by time and treatment group. Westley Croup Score, mean and 95% confidence 
interval by assessment and treatment group; circle and solid line represents dexamethasone, 
triangle and dashed lined represents low-dose dexamethasone, and square and dotted line rep- 


resents prednisolone. 


inclusion, and the number of patients 
who were excluded or who declined 
consent because data collection 
sheets were only retained for those 
who met enrollment criteria. Because 
of limited resources and challenging 
logistics (general population, ED 
sample), follow-up was not as robust 
as intended, with only ~70% of 
families contactable by phone. For the 
remaining 30%, we had to search ED 
attendance records for re-attenders 
diagnosed with croup; we were not 
able to assess all re-attendances in 
the study group; therefore, we may 
have missed those who re-attended 
ED with a different diagnosis, and we 
were unable to determine the rate of 
general practitioner (GP) re- 
attendance in this group. 


Our results indicate that it is 
acceptable to use any of the 3 
commonly used oral steroid regimes 
to treat croup in children. The vast 
majority (92%) of patients were 
successfully treated and discharged 
within 2 hours, improving from an 
average WCS of ~1.5 to ~0.5 over the 
first hour after treatment, with no 
differences between the 3 groups. 
The “ceiling effect” proposed by 
Geelhoed and Macdonald,'* whereby 
steroid doses higher than a certain 
threshold would not have any 
additional benefit, seems to be 
applicable in these patients, in line 
with several decades of experience 
with using low-dose dexamethasone 
for croup at our institution.* Our 
study revealed ED re-attendance 
rates similar to those from other 
studies, ”®16718 with no differences 
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TABLE 3 Re-attendance, Secondary Outcomes, and Adverse Events, by Treatment Group 


Variable Dexamethasone Low-Dose P Prednisolone P 
(Standard Treatment) Dexamethasone 
Follow-up status 
Phone call 286 (69.8%) 286 (69.8%) 1.00 282 (68.6%) 18 
ED records 124 (30.2%) 124 (30.2%) — 129 (31.4%) = 
Re-attendance to medical care 
No further treatment sought 557 550 20 322 Of 
Attended GP 49 44 — 58 — 
Re-attended ED 24 56 — 31 — 
Secondary outcomes 
Nebulized epinephrine 9 (2.2%) 12 (5.0%) 69 10 (2.5%) 99 
Endotracheal intubation 0 0 — 0 — 
Admission to intensive care 0 0 — 0 — 
Additional steroid dose(s)* 32 (11.3%) 42 (19.1%) .22 05 (18.9%) .02 
Total length of stay, min 125 120 06 126 19 
Length of stay >4 h 56 (8.8%) 29 (7.1%) 44 55 (8.5%) 99 
Adverse events 
Vomiting 16 (4.0%) 13 (3.3%) 19 13 (5.5%) 14 
Other 1 2 — 1 — 
Data are presented as count (%), median, or count, as appropriate. P values were calculated by using the x? test or, for total length of stay, by using a Kruskal-Wallis test. —, not 
applicable. 


a The denominator was reduced because of missing data. 


suggestive of a worse outcome for 
low-dose dexamethasone. One 
possible explanation would be that 
the steroid ceiling is at a dose higher 
than 0.15 mg/kg for a minority of 
patients. 


between groups for either GP or ED effect size was moderate (0.23), and 


croup re-attendance. even with the relatively reduced 


When comparing the groups at 2- and sample size available for the 3-hour 


3-hour clinical reviews, there appears 
to be a progressive divergence of the 
WCS for low-dose dexamethasone 
compared with dexamethasone. 
Although this difference reached 
Statistical significance (P = .042), the 


clinical review, the upper limit of the 
CI lies within the predefined 
noninferiority margin of a 0.5 
difference in the WCS. This result is 
broadly consistent with 
noninferiority, * although it is 


Duration of treatment has been raised 
by some authors’®?° who suggest 
that treatment with prednisolone 


TABLE 4 Additional Analysis and Consistency Analysis for Change in WCS, by Treatment Group 


Variable Dexamethasone (Standard Low-Dose 8 Coefficient or OR P Prednisolone 8 Coefficient or OR P 
Treatment) Dexamethasone (95% Cl) (95% Cl) 
WCS at 2 h 0.56 (0.86) (n = 107) 0.45 (1.05) (n = 126) .11 (—0.08 to 0.50) .41 0.41 (0.95) (n = .04 (—0.17 to 0.24) 72 


100) 
Change in WCS to 0.86 (1.42) (n = 107) 0.75 (1.50) (n = 126) — — 0.89 (1.27) (n= = — 
2h 100) 
WCS at 3h 0.15 (0.47) (n = 62) 0.59 (1.10) (n = 59) .25 (0.01 to 0.45) 04 0.24 (0.87) (n = 
79) 
0.55 (1.21) (n = — = 


04 (—0.17 to 0.24) 18 


Change in WCS to 0.48 (1.07) (n = 62) 0.65 (1.26) (n = 59) = —= 


ó h 79) 
Alternative 
outcomes 
1h 
Recovered 170 (44.7%) 175 (44.9%) 1.01 (0.76 to 1.54) .97 170 (43.8%) 0.96 (0.72 to 1.27) {T 
Improved 220 (60.5%) 250 (62.7%) 1.10 (0.82 to 1.49) .02 216 (58.9%) 0.95 (0.70 to 1.27) {1 
Ordinal — — 1.05 (0.72 to 1.54) .19 — 1.10 (0.75 to 1.62) 62 
outcome 
2h 
Recovered 215 (70.5%) 222 (74.0%) 1.20 (0.84 to 1.72) 62 202 (67.8%) 0.89 (0.63 to 1.25) AQ 
Improved 250 (95.8%) 259 (94.1%) 0.67 (0.29 to 1.52) 00 220 (96.9%) 1.30 (0.49 to 3.66) .60 
Ordinal — — 1.52 (0.76 to 3.10) 24 — 1.00 (0.48 to 2.09) .99 
outcome 


Data are presented as mean (SD) or count (%), as appropriate. B coefficients (95% CI) represent the differences between the treatment groups and standard treatment (dexamethasone) 
in the WCS at the follow-up assessment and were calculated by using linear regression adjusted for age, baseline WCS, and study center. ORs (95% Cl) were calculated by using logistic 
regression (recovered and improved) or ordinal regression adjusted for age, baseline WCS (as appropriate), and study center. P values were calculated by using linear regression, 
logistic regression, or ordinal regression, in line with coefficient reporting. —, not applicable. 
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Treatment 
Dexamethasone (standard) 


|-4-| Low-dose dexamethasone 


-®- Dexamethasone (standard) “= Prednisolone 


Treatment 


-—®- Low-dose dexamethasone 


-@- Prednisolone 


Total Length of Stay (Minutes) 


B with 95% CI 


Dexamethasone Low-dose Prednisolone 
(standard) dexamethasone 
Treatment 
FIGURE 4 
1 Hour 2 Hours 3 Hours Length of stay by treatment group. Violin plot 


of total length of stay by treatment group; 
Assessment points have a horizontal jitter for separation; 
FIGURE 3 mean and 95% confidence interval are in- 
dicated; display data truncated to length of 
stay less than 1000 minutes, 9 data points were 
removed from dexamethasone, 3 from low- 
dose dexamethasone, and 5 from prednisolone. 


Beta coefficient for low-dose dexamethasone and prednisolone groups relative to dexamethasone 
for WCS at the first three hourly assessments. Point represents the estimate of the difference in 
WCS at follow-up assessment between treatment groups (relative to dexamethasone [standard]), 
calculated via linear regression adjusted for age, baseline WCS, and study center; error bars 
represent 95% confidence interval around the beta estimate. 


should constitute multiple doses (3 days in the ED at the 3-hour mark. These enrolled patients. Dr Gareth Kameron 


in the study by Garbutt et al!) to cover nonresponders may have different helped with early study administration, 
the expected duration of the illness responses to steroid treatment, or including the study drug dispensing 
because prednisolone has a shorter they may require higher doses to mechanism, telephone follow-up of 
clinical duration of action.’” Our study effectively treat their croup. patients, and initial data collation. Dr 
was not designed to test different Dami Denbali helped with the telephone 
durations of treatment, but it did reveal CONCLUSIONS follow-up. Trial pharmacists Margaret 
that patients treated with a single dose Shave and Thanh Tan assisted greatly 
of prednisolone were statistically more Oral steroids are an effective with medications management. We also 
likely (P = .02) to receive additional treatment of croup, and the type of thank ED doctors and nurses at Princess 
doses of the steroid than those treated steroid seems to have no clinically Margaret Hospital and Joondalup Health 
with dexamethasone. significant impact on efficacy, both Campus for consenting and enrolling 
acutely and during the week after patients during their busy shifts. 


One suggestion for further study 
relates to the apparent weakening 
performance for low-dose 


treatment. Children treated with 
prednisolone initially are more likely 
to require additional doses to cover 


dexamethasone (0.15 mg/kg) at the the duration of the illness. PAREN 

3-hour assessment. This effect may be CI: confidence interval 

due to a small number of patients ED: emergency department 
who do not respond to oral steroid ACKNOWLEDGMENTS GP: general practitioner 
treatment within 1 to 2 hours, We gratefully acknowledge Sharon OR: odds ratio 

constituting a treatment-resistant O’Brien (research assistant) who WCS: Westley Croup Score 
cohort; <4% of our patients were still conducted the telephone follow-up of 
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Differences in Parent-loddler 
Interactions With Electronic Versus 
Print Books 


Tiffany G. Munzer, MD,? Alison L. Miller, PhD,®* Heidi M. Weeks, PhD,‘ Niko Kaciroti, PhD,°® Jenny Radesky, MD? 


OBJECTIVES: Previous research has documented less dialogic interaction between parents and 
preschoolers during electronic-book reading versus print. Parent-toddler interactions around 
commercially available tablet-based books have not been described. We examined parent- 
toddler verbal and nonverbal interactions when reading electronic versus print books. 


METHODS: We conducted a videotaped, laboratory-based, counterbalanced study of 37 parent- 
toddler dyads reading on 3 book formats (enhanced electronic [sound effects and/or 
animation], basic electronic, and print). We coded verbalizations in 10-second intervals for 
parents (dialogic, nondialogic, text reading, format related, negative format-related directives, 
and off task) and children (book related, negative, and off task). Shared positive affect and 
collaborative book reading were coded on a scale of 1 to 5 (5 = high). Proc Genmod and Proc 
Mixed analyzed within-subjects variance by book format. 


RESULTS: Parents showed significantly more dialogic (print 11.9; enhanced 6.2 [P < .001]; basic 
8.3 [P < .001]), text-reading (print 14.3; enhanced 10.6 [P = .003]; basic 14.4 [P < .001]), off- 
task (print 2.3; enhanced 1.3 [P = .007]), and total (29.5; enhanced 28.1 [P = .003]; basic 29.3 
[P = .005]) verbalizations with print books and fewer format-related verbalizations (print 1.9; 
enhanced 10.0 [P < .001]; basic 8.3 [P < .001]). Toddlers showed more book-related 
verbalizations (print 15.0; enhanced 11.5 [P < .001]; basic 12.5 [P = .005]), total 
verbalizations (print 18.8; enhanced 13.8 [P < .001]; basic 15.3 [P < .001]), and higher 
collaboration scores (print 3.1; enhanced 2.7 [P = .004]; basic 2.8 [P = .02]) with print-book 
reading. 

CONCLUSIONS: Parents and toddlers verbalized less with electronic books, and collaboration was 
lower. Future studies should examine specific aspects of tablet-book design that support 
parent-child interaction. Pediatricians may wish to continue promoting shared reading of 
print books, particularly for toddlers and younger children. 


Oo 


WHAT’S KNOWN ON THIS SUBJECT: When preschoolers read 
electronic books with parents, parents may show less dialogic 


“Department of Pediatrics, Medical School, Departments of Health Behavior and Health Education, “Nutritional 
Sciences, and “Biostatistics, School of Public Health, and “Center for Human Growth and Development, University 


of Michigan, Ann Arbor. Michigan reading, and talk is often focused on the technology. It is not 
known whether toddler-parent interactions differ when reading 
Dr Munzer conceptualized and designed the study, coordinated and supervised data collection, commercially available electronic books compared with print. 


drafted the initial manuscript, and reviewed and revised the manuscript; Dr Miller conceptualized 
and designed the study and reviewed and revised the manuscript; Dr Weeks conducted the data 
management, processing, and analyses and reviewed and revised the manuscript; Dr Kaciroti 
created the data analysis plan, conducted the data analysis, and reviewed and revised the 
manuscript; Dr Radesky conceptualized and designed the study, coordinated and supervised data 
collection, and reviewed and revised the manuscript; and all authors approved the final manuscript 
as submitted and agree to be accountable for all aspects of the work. 


WHAT THIS STUDY ADDS: Parents engaged in more dialogic 
reading with fewer technology-related verbalizations and more 
parent-toddler verbalizations with print books compared with 
electronic books. Print books elicited a higher quality of parent- 
toddler collaborative reading experience compared with 
electronic books. 


DOI: https://doi.org/10.1542/peds.20 18-2012 To cite: Munzer TG, Miller AL, Weeks HM, et al. Differences 
Accepted for publication Dec 10, 2018 in Parent-Toddler Interactions With Electronic Versus 
Print Books. Pediatrics. 2019;143(4):e20182012 
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Shared book reading is 1 of the most 
important developmental activities 
parents can engage in with their 
children.’ Shared book reading 
exposes children to more 
sophisticated speech and 
knowledge,” and provides unhurried 
time to build attachment,” in turn 
promoting executive functioning 
skills.” Nonverbal interactions during 
shared book reading, such as parental 
warmth and child enthusiasm, foster 
interest in reading and are associated 
with improved literacy later in life.°” 
In particular, parent dialogic reading 
practices (comments and questions 
that go beyond the written word and 
connect the story to child 
experiences) are believed to promote 
child expressive language, 
engagement, and literacy.°"*” 


With rapid increases in electronic- 
book and mobile-device 
ownership,” a growing amount of 
children’s reading is taking place 
electronically on electronic readers or 
tablets. However, pediatricians are 
unsure whether to promote their use 
because previous studies suggest 
both benefits and drawbacks to 
electronic reading for preschoolers 
and older children.’*** Previous 
literature has shown that electronic 
books may facilitate engagement, 
particularly among reluctant 
preschoolers and kindergarteners 
who are learning to read.’* *° Certain 
embedded tools, such as dictionaries, 
may improve vocabulary and story 
comprehension in 
kindergarteners.1™1719 However, 
preschoolers and kindergarteners 
also reproduced fewer narrative 
details’? and sequenced story events 
with lower accuracy after reading 
enhanced electronic books compared 
with print books.’*”° Lower 
comprehension during electronic- 
book reading may be due to less adult 
verbal elaboration’* and scaffolding*® 
and extraneous “hot-spot” 
enhancements, which may distract 
from story content.” Yet, adult verbal 
elaboration and parental scaffolding 


is crucial for young children’s 
learning,** particularly regarding 
digital media.**-** 


An existing gap in knowledge is how 
toddlers and parents interact around 
electronic books. Developmentally, 
toddlerhood (~24-36 months old) is 
characterized by emerging language 
and social-emotional skills as well as 
immature executive functioning skills. 
These developmental differences may 
make toddlers particularly 
susceptible to the distractions?” in 
enhanced electronic books. 
Additionally, because of their 
immature memory flexibility, toddlers 
depend more on adult scaffolding to 
transfer information from digital 
media to the real world,*°*’ have 
more difficulty learning information 
presented in digital media compared 
with in-person interactions,“° and 
retain information better when digital 
media are viewed with an adult.** 
Only 1 electronic-book study has 
been conducted in toddlers, finding 
that toddlers remembered a novel 
word better on an electronic book 
compared with print, but parents 
read the text and pointed more when 
interacting over print.*’ This study 
used an electronic book without 
digital enhancements that was not 
commercially available; therefore, 
results do not generalize to tablet- 
based books available to families.** 
To our knowledge, no studies have 
examined dialogic and nonverbal 
interactions between parents and 
toddlers when reading 

electronic books. 


For pediatric providers to make 
informed decisions about 
recommending electronic books, 
more needs to be known about 
differences in parent-child 
interactions during these types of 
reading encounters with toddlers, 

a developmental range that is 
underrepresented in current 
literature. In this study, we aim to 
address these gaps by examining the 
frequency of parent verbalizations 
that are important to early language 


and literacy (eg, dialogic reading), 
child verbalizations, and quality of the 
shared book-reading experience 
during the reading of commercially 
available electronic and print books. 


METHODS 


Study Design 


We conducted an experimental, 
laboratory-based study consisting of 
a video-recorded free play, book- 
reading protocol, and surveys lasting 
~75 minutes. Toddler-parent dyads 
were assigned to 1 of 36 
counterbalanced book-format orders. 
Parents were compensated $50 for 
participating. The University of 
Michigan Institutional Review Board 
approved this study. 


Participants 


We recruited 37 parent-toddler dyads 
from the University of Michigan 
online research registry 
(UMhealthresearch.org) and 
community-based settings, including 
pediatric offices, child care centers, 
and community centers. To not bias 
recruitment toward parents with 
particular digital media views, 
language stated generically that the 
study involved coming to the 
University of Michigan, where “you 
and your child would be videotaped 
while playing with toys and books.” 
Parents contacted researchers via the 
research registry, e-mail, or phone 
and underwent phone-based 
screening. Inclusion criteria were as 
follows: (1) child age 24 to 36 
months, (2) child did not have 

a developmental delay or serious 
medical condition, (3) parent read 
English sufficiently to complete 
questionnaires and consent, (4) 
parent was a legal and/or physical 
custodial guardian, and (5) parent 
and child did not have uncorrected 
hearing or vision impairments. 


Procedure 


At the study visit, parents provided 
written informed consent. The 
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laboratory room was set up to 
approximate a living room and 
contained a 1-way mirror, couches, 3 
books in boxes (2 tablet books and 1 
print book), and video cameras. 


Participants first completed a 5- 
minute, video-recorded free play with 
nondigital toys. They then completed 
a random, preassigned reading 
activity with an enhanced electronic 
book, a basic electronic book, and 

a print book occurring in 
counterbalanced order (Fig 1). 
Figure 2 includes 1 example 
permutation. Books were placed in 
open boxes labeled 1 to 3 out of 
children’s reach. Parents received 
instructions to start with the book in 
box 1, that they have “5 minutes to 
look at it,’ and to complete books 
sequentially as prompted. 


Book Formats 


Three Mercer Mayer “Little Critter” 
books (Just Grandma and Me, All by 
Myself, and Just a Mess) were chosen 
because of their similar length, 
reading difficulty, and availability in 
all 3 formats. Print books were 8X8- 
inch softcovers. Basic electronic-book 
capabilities allowed for swiping to 
turn the pages and tapping 
illustrations to elicit the appearance 
of words but without autonarration 
or additional features, such as sound 
effects. Enhanced electronic books 
contained audiovisual hot spots: 
tapping illustrations would result in 
the appearance and narration of the 
word (eg, tapping a seagull picture 
resulted in the appearance and 


Book format A 


Book format B 


Basic 


All by Myself 


Enhanced 


Just Grandma and Me 


Print 


Just a Mess 


5 minutes 


FIGURE 2 


5 minutes 


5 minutes 


Sample of 1 book-reading permutation completed by a participant. 


narration of the word “seagull”) 
coupled with sound effects. Tapping 
other pictures or turning a page 
produced a sound effect (eg, tapping 
a dog would produce the sound of 

a dog panting, and turning the page to 
a beach produced sounds of ocean 
waves). Although autonarration of the 
story was disabled on both 
electronic-book formats, tapping and 
holding down an individual sentence 
in the enhanced electronic book 
would narrate that text, but this 
feature was only briefly used by 2 
dyads. Basic and enhanced electronic 
books were preloaded on a 10-inch 
Samsung Galaxy tablet computer, 
which contained no other 
applications. Parents received 
instruction to select “read it myself” 
such that the electronic book was not 
narrating the book text. 


Survey Measures 


Parents completed surveys regarding 
covariates for potential inclusion in 
statistical models, including 
demographic information (parent age, 
sex, educational attainment, 
household income, race and/or 
ethnicity, relationship to child, and 
marital status; child’s age, sex, 
ethnicity, and prematurity) and 
standardized measures of child 


Book format C 


Title 1 


5 minutes 


FIGURE 1 


Title 2 


5 minutes 


Title 3 


5 minutes 


Reading protocol. The reading protocol consisted of a preassigned sequential reading activity of an 
enhanced electronic book, a basic electronic book, and a print book occurring in a counterbalanced 
fashion in 1 of 6 book-format permutations: (1) enhanced, basic, print; (2) enhanced, print, basic; (3) 
basic, enhanced, print; (4) basic, print, enhanced; (5) print, basic, enhanced; or (6) print, enhanced, 
basic. Within each book-format permutation, the order of 3 different book titles was counter- 
balanced, achieving a total of 36 unique permutations. Thus, all participants read the same 3 books, 
but not all books were read in the same format or order across participants. 


language, social-emotional 
development, and digital media-use 
practices. 


The MacArthur-Bates Communicative 
Developmental Inventory (CDI) short 
form assessed toddler language 
development. This 100-word 
validated*® and reliable’? vocabulary 
checklist generated a percentile score 
from total words produced and 
accounted for age.** 


The Brief Infant-Toddler Social and 
Emotional Assessment (BITSEA) is 

a validated?’ and reliable?” 42-item 
questionnaire that screens for child 
social-emotional problems. Parents 
rated items on a 3-point Likert scale 
generating the Problem and 
Competence subscales (Cronbach 

a = 0.68 and 0.58, respectively). 


Standardized questions assessed the 
frequency of home child digital media 
use (including tablet, smartphone, 
and electronic-book usage) and 
parental mediation strategies 
(instructive, restrictive, and 
coviewing).°* 


Coding Parent-Toddler Verbal 
Interactions 


We developed a verbal coding scheme 
based on previous literature on 
dialogic reading® and shared 
electronic-book reading.’* For each 
10-second interval, researchers coded 
1 for a specific verbalization 
occurring or 0 for not occurring; 
interval verbalization counts were 
summed within each 5-minute book 
condition. Verbalization categories 
were not mutually exclusive; parents 
and children could have >1 type per 
10-second interval, although each 
sentence was only coded in 1 
category. Please see the coding 
definitions in Tables 1 and 2 for 
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TABLE 1 Coding Definitions and Examples of Parent Verbalizations 


Definition Examples Cohen's 
K 
Dialogic Dialogic reading techniques often prompt a child to expand and “What’s happening here?” “What did they 0.77 
elaborate on concepts related to the story. These were do next?” “What did you think about 
defined as follows: parent asks open-ended question, expands that book?” “Remember when you went 
on an idea the child has, repeats what the child says, or to the beach with Dad?” Child says, 
relates the story content to the child’s experience. “Here is a wagon, and parent replies, 
“a big, red wagon.” 
Nondialogic Nondialogic reading techniques were related to story content “What is that?” “Show me the cat.” “Look 0.74 
but have not been previously shown to elicit the same at this!” “There’s Grandma!” 
quantity of child verbalizations as dialogic verbalizations. 
These were defined as follows: parent labels something, asks 
a simple question requiring only a name or label, makes 
a pointing request of the child, makes an attention prompt, or 
talks about the process of reading. 
Text read Parent reads directly from the book text. “We went to the beach, just grandma and 0.86 
me.” 
Format related These are verbalizations that are related to the book format. “Great job, you’re turning the page!” “Can 0.84 
Parent comments on, asks a question about, or adds | hold the book or tablet?” “Go ahead 
a directive regarding an aspect of the print or tablet and turn the page.” “You can push the 
interface. button here.” “Swipe with your finger.” 
Negative format-related Parent makes a negative directive that is related to the book “You can't keep pressing the back 0.80 
directives format. For instance, the parent tells the child not to do button.” “Don’t turn the page.” “Don’t 
something related to how the book or tablet functions. rip the book.” “Don’t turn the volume 
up.” “Don’t touch that button.” 
Off task These are unrelated to the book content or book format and “You can have your goldfish later.” “We 0.79 


include all other parent verbalizations that are not 
categorized as above. 


parent and child verbalizations. 
Parent and child utterances were 
independently summed to calculate 
total verbalizations. Undergraduate 
students blinded to the hypothesis 
coded to reliability with Cohen’s k of 
at least 0.70. 


Coding Parent-Toddler Nonverbal 
Interactions 


We developed 2 global coding 
schemes based on existing literature 
on shared print-book reading to 
assess parent-toddler nonverbal 
interactions: shared positive affect’? 
and collaborative book-reading 
experience.** *° Codes were applied 


on the basis of the full 5 minutes per 
book condition on a scale of 1 to 5 
(Table 3). The 5-minute free-play 
session was also coded for shared 
positive affect and examined as 

a potential covariate representing 
baseline parent-child interaction 
quality. 


Analysis 


We conducted Poisson regressions 
using Proc Genmod to compare each 
verbal outcome by book format, 
adjusting for total elapsed time, given 
the occasional variation in reading 
duration. Proc Mixed was used to 
compare differences in positive affect 


TABLE 2 Coding Definitions and Examples of Child Verbalizations 


are going to the store after this.” 


and collaborative book reading by 
book format. All models included 

a repeated measures statement to 
allow for within-subjects comparison 
of verbal and nonverbal outcomes by 
book-format condition. Although the 
counterbalanced design accounted 
for between-subjects variance in 
factors known to influence book- 
reading behaviors, such as 
sociodemographic characteristics, we 
included covariates in final models 
with P < .05 to improve model fit (eg, 
order of book presentation, parent 
income, race and/or ethnicity, child 
sex, CDI or BITSEA score, and home 
media practices). A sensitivity 


Definition Examples Cohen's 
K 
Book related Child labels a picture, answers a parent question, repeats what “I want to read this.” “| press the button.” 0.81 
the parent is saying, or talks about a function of the book. “Look, a spider!” 
Negative Child says no or makes a comment in a defiant or negative “No, Daddy, | do it.” “I hold it.” 0.71 
manner. 
Other These are verbalizations that do not fall into the above “Can | have water?” “I want to go home.” 0.72 
categories. Unintelligible utterances that are not clearly 
related to the book were also included. 
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TABLE 3 Nonverbal Coding Definitions 


Definition 


Shared Positive Affect 


Collaborative Book Reading 


Quantity of shared enjoyment between dyad 


Intraclass correlation 0.84 


Code 1 A score of 1 was marked by little positive shared affect or 
enjoyment, several instances of negative affect that occur 


more frequently than instances of positive affect, and/or the 
child having a tantrum or refusal of prolonged duration or 


high frequency. 


Code 3 A score of 5 was marked by small-to-moderate amounts of 


positive affect between the dyad with brief but unsustained 
instances of negative affect, or the dyad may be primarily 
affectively neutral. 


Code 5 A score of 5 was marked by frequent displays of shared 


positive affect with the dyad showing definite pleasure with 
each other (eg, high frequency of smiling, laughing, praise, 


and warmth). 


analysis excluding 1 participant who 
cried during the entirety of 1 book- 
reading condition did not reveal 
differences; therefore, all participants 
were included. All analyses were 
completed in SAS 9.4 (SAS Institute, 
Inc, Cary, NC). 


RESULTS 


As shown in Table 4, children were 
29.2 months old, and parents were 
33.5 years old. Of the parents, 81% 
were mothers, 76% had a 4-year 
college degree or more, and 89% 
were married. Of the children, 54% 
were boys, 57% were non-Hispanic 
white, 16% were non-Hispanic 
African American, and 27% were of 
other race and/or ethnicity. 


Figure 3 shows the number of 
intervals containing each type of 
parent verbalization. Parent dialogic 
verbalizations were greater with 
print (11.9 intervals [SE = 1.1]) 
versus either enhanced electronic 
(6.2 intervals [SE = 0.7]; P < .001) or 
basic electronic books (8.3 intervals 
[SE = 0.9]; P < .001). Parent 
nondialogic verbalizations were 
greater with print (17.7 [SE = 0.7]) 
versus basic electronic books (15.7 
[SE = 0.8]; P = .008). Parents read the 
book text more with print (14.3 [SE = 
1.0]; P = .003) or basic electronic 
(14.4 [SE = 1.1]; P < .001) compared 


Quality of shared reading experience 


0.75 
A score of 1 was marked by greater distance between the 


parent and child, the parent making few attempts to engage 
the child or being overly directive and/or intrusive, or the 
child missing social bids from the parent or being 
confrontational and/or defiant. 


A score of 5 was marked by some instances of close dyad 


proximity with some instances of greater distance between 
them, some attempts of parent-child engagement but less in 
frequency than a code 4 or 5, and/or the dyad seeming more 
focused on the reading task than on each other. 


A score of 5 was marked by the dyad being comfortably nestled 


together with a shared view of the book, a highly responsive 
parent, and an actively engaged child who exhibits minimal 
defiance. 


with enhanced electronic books (10.6 
[SE = 0.9]). Parents made fewer 
format-related and negative format- 


TABLE 4 Participant Characteristics (N = 37) 


related directives when engaging over 
print books versus enhanced or basic 
electronic books (Fig 3). Parents had 


Sample Result 
Child age, mo, mean (SD) 29.2 (4.2) 
Parent age, y, mean (SD) 33.0 (4.0) 
Parent relationship to child, n (%) 
Mother 30 (81) 
Father 7 (19) 
Child sex, n (%) 
Boys 20 (54) 
Girls 17 (46) 
Child race and/or ethnicity, n (%) 
White, non-Hispanic 21 (57) 
African American, non-Hispanic 6 (16) 
Hispanic or other 10 (27) 
Parent education, n (%) 
Some college courses 4 (11) 
2-y college degree 5 (15) 
4-y college degree 14 (38) 
More than 4-y college degree 14 (58) 
Parent marital status, n (%) 
Single 4 (11) 
Married 33 (89) 
Child has used tablet to read a book, n (%) 
Almost never 25 (62) 
Rarely 2 (5) 
Occasionally 4 (11) 
Often 6 (16) 
Most of the time 2 (5) 
Daily time spent reading books together, n (%) 
Not used 8 (22) 
<30 min 16 (45) 
50 min—1 h 9 (24) 
1-2 h 3 (8) 
5-4 h 1 (3) 
CDI percentile, mean (SD) 52.9 (33.4) 
BITSEA Problem subscale, mean (SD) 6.7 (3.8) 
BITSEA Competence subscale, mean (SD) 19.1 (2.2) 
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FIGURE 5 


Adjusted means for the presence of parent verbalizations occurring with enhanced electronic, basic 


electronic, and print books. 


more total verbalizations when 
interacting over print books (29.5 [SE 
= 0.2]) versus enhanced electronic 
books (28.1 [SE = 0.4]; P = .003) and 
more so over basic electronic books 
(29.3 [SE = 0.3]; P = .005) versus 
enhanced electronic books. 


Figure 4 includes the number of 
intervals containing each type of 
toddler verbalization. Toddlers’ book- 
content verbalizations were greater 
with print (15.0 [SE = 1.2]) versus 
either enhanced electronic (11.5 [SE = 
0.9]; P < .001) or basic electronic 
books (12.5 [SE = 1.1]; P = .005). 
Toddlers had no differences in 
negative verbalizations across all 
formats but had more off-task 
verbalizations with print (2.3 [SE = 
0.3]) versus enhanced electronic 
books (1.3 [SE = 0.3]; P = .007). Total 
toddler verbalizations were greater 
with print (18.8 [SE = 1.1]) versus 
either enhanced electronic (13.8 [SE = 
0.9]; P < .001) or basic electronic 
books (15.3 [SE = 1.0]; P < .001). 


Figure 5 includes nonverbal 
outcomes by book format. Shared 
positive affect was similar across all 
book formats. Dyads’ collaborative 


* P< Dor P< 01; ** P< 001. 


book-reading scores were higher with 
print (3.1 [SE = 0.2]) versus either 
enhanced electronic (2.7 [SE = 0.2]; 
P = .004) or basic electronic books 
(2.8 [SE = 0.2]; P = .02). 


DISCUSSION 


Developmental benefits of shared 
book reading have been attributed to 
the quality of parent-child 
interactions occurring around books, 
particularly in prereaders such as 
toddlers, who rely heavily on parents 
to understand story content.® These 
interactions include the quantity of 
words spoken, how parents tailor 
content to children’s experiences to 
support learning, and asking open- 
ended questions to promote child 
expressive language.”? Our findings 
suggest that high-quality dialogic 
practices are less common, and 
parents and toddlers speak less 
overall and in a less collaborative 
manner, when reading electronic 
books compared with print. Parents 
read the text less in enhanced 
electronic books, making more 
format-related comments and 


negative directives when reading 
electronic books. 


Similar to previous studies in 
preschoolers,’ we found that 
electronic-book enhancements were 
likely interfering with parents’ ability 
to engage in dialogic reading. 
Dialogic, parent-guided conversation 
promotes toddler expressive- 
language development and supports 
preliteracy skills, which are crucial 
for independent reading,® far more 
than reading only text or making 
simple (nondialogic) comments, 
although these are also important.’ 
Parents strengthen their children’s 
ability to acquire knowledge by 
relating new content to their 
children’s lived experiences.” *’6 
There is a large body of literature 
showing that this type of adult 
scaffolding is especially important for 
toddlers to transfer information from 
digital media to the real world 
because toddlers in particular learn 
and retain novel information better 
from in-person interactions than from 
digital media.*7-**7°2”37 However, 
such practices occurred less 
frequently with electronic books, 
which raises the question of whether 
electronic books have lower 
educational potential for toddlers. 


Parents also asked fewer simple 
questions, commented about the 
storyline less, and read less during 
electronic-book conditions compared 
with print. These behaviors are 
important because they promote 
child receptive language by exposing 
children to novel vocabulary and 
more complex syntax than 
conversations occurring during daily 
activities.’ 


Even interactions over basic 
electronic books contained fewer 
dialogic and total parent 
verbalizations compared with print, 
suggesting that affordances of the 
tablet (and not only the interactive 
design) may be influencing parents’ 
behavior. Parents and children may 
conceptualize tablets as being 
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Adjusted means for the presence of toddler verbalizations occurring with enhanced electronic, basic 
electronic, and print books. * P < .05; ** P < .01; ** P < .001. 


individually used rather than shared 
objects. Indeed, 1 study revealed that 
children tend to create solitary spaces 
when engaging in tablet play 
compared with traditional toy play, 
leaving less space for their parents to 
coview and ignoring parent bids for 
attention.”® Similarly, parents 
reported a sense of pride and relief 
when their children independently 
engaged with a tablet device without 
help,*’ and we acknowledge that this 
independence may be perceived as 

a potential benefit of electronic 
books. We hypothesize that the tablet 
itself may reduce opportunities for 


parent-child interactions during book 
reading. 


Children changed their behavior as 

a function of book format, verbalizing 
more when reading the print book. 
This finding may be related to greater 
parent dialogic reading with print 
books versus electronic books, which 
provides positive reinforcement for 
toddler speech. Children’s tendency 
to become occupied in repeated 
tapping or swiping on electronic 
books may also have supplanted 
speech production. Repetitive tapping 
and swiping may not constitute 


sufficient engagement to learn new 
concepts because it is thought to 
represent cause-and-effect play 
rather than “minds-on” activity.” 
True meaningful engagement (active 
involvement occurring in a rich social 
context without distractions) fosters 
the most effective learning from 
media.** Previous research in 
preschoolers supports this concept 
because distracting digital 
enhancements interfere with parent 
scaffolding, which leads to reduced 
child story comprehension and fewer 
child verbalizations.’*** 
Opportunities to practice expressive 
language, such as those occurring 
with print books, are important 
because early language skills strongly 
predict future linguistic and cognitive 
aptitude in school.*” 


The high frequency of format-related 
verbalizations (eg, directing the child 
to turn the page) observed during 
both electronic-book conditions may 
displace book-related verbal 
exchanges that dyads engaged in with 
print books. This is consistent with 
previous research: although parents 
showed the same number of verbal 
exchanges with preschoolers around 
electronic versus print books, 
exchanges tended to be related to 
technology rather than story 
content.’* It is possible that parents 
made more format-related 
verbalizations to orient their children 
to a new experience because 
electronic books were novel to 62% 
of the children in this sample; 
however, 79% had previously played 
with tablets and/or mobile devices. 
The negative and directive nature of 
parent format-related verbalizations 
may indicate a need for more 
behavioral management with 
electronic books compared with print. 


Parents and children had more off- 
task verbalizations with the print 
book versus electronic book, which is 
similar to previous studies in 
preschoolers.’* This could be related 
to persuasive tablet-design features, 
which may command parent and 
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child attention, at the expense of 
attending to one another, an effect 
that is known to occur with 
traditional screen media, such as 
television.**~*° It is challenging to 
discern whether this attentional focus 
resulted in improved learning 
because toddler reading 
comprehension is difficult to assess. 
However, as mentioned above, 
toddlers may be engaged in ways that 
may be less educationally enriching 
when reading electronic compared 
with print books. 


Our study was the first to examine 
nonverbal aspects of electronic-book 
reading in toddlers both through 
shared positive affect and 
collaborative book reading. These 
nonverbal behaviors during reading 
are important because they foster 

a love of reading’ and promote secure 
parent-child attachment,’ which has 
been implicated in resilience,** 
physical health,*” and quality of 
future relationships.*° Comparable to 
previous studies in preschoolers,“ 
dyads with high shared positive affect 
consistently showed this across all 


formats, suggesting that electronic 
books may be equally enjoyable for 
dyads despite other limitations. The 
quality of collaborative reading was 
likely lower for electronic books 
because observationally, parents and 
toddlers frequently sat separately, 
could not easily view the book, or 
appeared to struggle for tablet 
possession. These behaviors during 
tablet-based play are documented in 
recent work®® and merit further 
study. Our results may explain 
previous findings that parents report 
preferring shared reading over print 
versus electronic books with 
toddlers.*” 


Limitations include the small sample 
size from 1 geographic area, the use 
of only 1 type of book-reading 
application (which limits 
generalizability), and that the age 
range of our study sample precluded 
assessment of reading 
comprehension.*® Strengths include 
experimental methodology, the use of 
commercially available books 
available in 3 formats, a diverse 
sample, and the within-subjects 


design, which allowed for direct 
comparison of the reading experience 
within each dyad. Future studies 
should consider other facets of 
nonverbal interactions or moderating 
effects of dyad characteristics, such as 
parent literacy level, child 
temperament, or home media-use 
practices. Replication of this study by 
using different applications (this 
application had a particular set of 
enhancements) in other contexts, 
such as home or school settings, is 
necessary. Although parent-child 
interactions are critical for toddler 
learning, directly examining toddler 
learning from print versus electronic 
books is another important area for 
future work. 


CONCLUSIONS 


Given the decreased quantity of 
parent-child verbalizations and 
quality of interactions occurring with 
the electronic books that we studied, 
pediatricians may wish to 
recommend print books over 
electronic books with distracting 
features for parent-toddler shared 
reading. In considering affordances of 
electronic books that promote 
learning, software designers should 
limit irrelevant audiovisual 
enhancements for toddlers. Parents 
reading electronic books with 
toddlers should consider engaging as 
they would with print and minimize 
focus on elements of the technology 
itself. 
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BACKGROUND AND OBJECTIVES: Annually, several hundred infant deaths occur in sitting devices (eg, 
car safety seats [CSSs] and strollers). Although American Academy of Pediatrics guidelines 
discourage routine sleeping in sitting devices, little is known about factors associated with 
deaths in sitting devices. Our objective was to describe factors associated with sleep-related 
infant deaths in sitting devices. 


METHODS: We analyzed 2004-2014 National Center for Fatality Review and Prevention data. The 
main outcome was sleep location (sitting device versus not). Setting, primary caregiver, 
supervisor at time of death, bed-sharing, and objects in the environment were compared by 
using x^ tests and multivariable logistic regression. Descriptive statistics of additional possible 
risk factors were reviewed. 


RESULTS: Of 11 779 infant sleep-related deaths, 348 (3.0%) occurred in sitting devices. Of deaths 
in sitting devices, 62.9% were in CSSs, and in these cases, the CSS was used as directed in 
<10%. Among all sitting-device deaths, 81.9% had =1 risk factor, and 54.9% had =2 risk 
factors. More than half (51.6%) of deaths in CSSs were at the child’s home. Compared with 
other deaths, deaths in sitting devices had higher odds of occurring under the supervision of 
a child care provider (adjusted odds ratio 2.8; 95% confidence interval 1.5-5.2) or baby-sitter 
(adjusted odds ratio 2.0; 95% confidence interval 1.3-3.2) compared with a parent. 


CONCLUSIONS: There are higher odds of sleep-related infant death in sitting devices when a child 
care provider or baby-sitter is the primary supervisor. Using CSSs for sleep in nontraveling 
contexts may pose a risk to the infant. 
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Sleep-related infant death is 

a categorical term that includes 
sudden infant death syndrome (SIDS), 
accidental suffocation and 
strangulation in bed (ASSB), and 
other ill-defined deaths.’ 
Approximately 3700 infants annually 
suffer sleep-related infant death, 
making this the leading cause of 
postneonatal (28 days-1 year old) 
infant mortality.” 


In an effort to decrease the rate of 
sleep-related deaths, the American 
Academy of Pediatrics (AAP) has 
released recommendations for a safe 
infant sleep environment.’ Those 
recommendations include placing the 
infant in a supine sleep position, 
using a firm sleep surface, room- 
sharing without bed-sharing, avoiding 
the use of soft bedding and 
overheating, and particularly 
regarding sitting devices, refraining 
from the use of sitting devices, such 
as car Safety seats (CSSs) and 
strollers for routine sleep.’ 


Sitting and carrying devices (such as 
CSSs, strollers, infant seats, bouncers, 
and swings) were initially designed 
primarily for the transporting, 
feeding, and play of infants and young 
children. Despite the intended 
purpose of these devices (eg, 
transportation, feeding, and playing), 
infants spend a significant amount of 
time in sitting devices for other 
reasons. One observational study of 
187 infants found that the mean time 
an infant spent in a sitting device was 
5.7 hours (range 0-16 hours) per 
day.” In some cases, these devices are 
used as a Sleep alternative to a crib or 
bassinet.* © 


When placed in a sitting device for 

a reason other than its intended 
purpose (ie, not being used for 
transportation, feeding, or playing), 
infants are at risk for injury, including 
falls from the devices, falls from 
elevated surfaces on which the device 
has been placed, and injuries when 
the device flips over on a soft 
surface.’ *° Additionally, in 1 


retrospective study of sleep-related 
deaths of young (<2 years old) 
children occurring in sitting devices, 
many parents used the device as 

a place for their children to sleep; 16 
of the 18 deaths involving a CSS 
occurred outside a motor vehicle, 
suggesting use of the CSS other than 
for its intended purpose.” Similarly, 
a UK study of 14 CSS-related infant 
deaths between 1999 and 2011 found 
that most deaths occurred when the 
CSSs were not being used to transport 
an infant in a motor vehicle and that 
most of these deaths occurred in 
infants <2 months of age.” 


Reports of infant deaths in sitting 
devices have thus far included 

a limited number of deaths and 
primarily focused on the intended use 
of the sitting device at the time of 
death. Little is known about other 
factors surrounding infant deaths in 
sitting devices. In this study, we 
analyzed data from a large US case- 
reporting system to better 
understand the factors associated 
with sleep-related infant deaths in 
sitting devices. 


METHODS 


Data Source 


Data for this study were obtained 
from the National Center for Fatality 
Review and Prevention (NCFRP). The 
NCFRP, funded by the Maternal and 
Child Health Bureau, Health 
Resources and Service 
Administration, US Department of 
Health and Human Services 
(UG7MC28482), and US Centers for 
Disease Control and Prevention, is 

a Web-based system used for 
systematically collecting, analyzing, 
and reporting data on infant and child 
deaths.’* After approval and consent 
from the 45 states participating in the 
NCFRP, we received a deidentified 
data set of infant deaths occurring 
during sleep or in a sleep 
environment from 2004 to 2014. The 
NCFRP Child Death Review Case 


Reporting System has been 
previously described.**~** 


Study Subjects 


The NCFRP provided data on all 
infant (5365 days of age) deaths 
occurring during sleep. Deaths that 
could be explained by nonsleep- 
related causes, such as homicide or 
infection, and deaths missing sleep 
location were excluded. By using data 
regarding cause-of-death 
determination, cases were assigned to 
1 of 3 causes of death: SIDS or sudden 
unexpected infant death (SUID), 
ASSB, and unknown or undetermined. 


Sleep Location and Setting of Death 


We categorized sleep location as the 
location in which the infant death 
occurred; deaths were dichotomized 
into those in sitting devices (CSSs, 
strollers, bouncers, swings, and other 
infant seats) or not in sitting devices 
(crib, bassinet, playpen, adult bed, 
and couch or chair). Infant slings and 
carriers and nursing pillows were not 
categorized as sitting devices. Setting 
of death was defined as the 
environment in which the infant 
death occurred and was categorized 
as follows: child’s home; relative’s or 
friend’s home; foster home; child care 
center or baby-sitter’s home; vehicle 
in motion or temporarily stopped or 
parked; shopping or errands or car 
seat or stroller on a walk; motel, 
recreational vehicle or trailer, or park; 
and other. 


Primary Caregiver and Primary 
Supervisor 


The primary caregiver was defined as 
the person who was responsible for 
the infant on a daily basis and was 
categorized as a parent or guardian, 
foster parent, relative or friend, or 
other. Primary supervisor was 
defined as the person who was 
responsible for the infant at the time 
of death and was categorized as 

a parent or guardian, relative, friend, 
child care provider (licensed or 
unlicensed), baby-sitter, other, or 
unknown. 
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Data Analysis 

Data analysis was performed by using 
SAS (SAS Institute, Inc, Cary, NC) and 
SPSS (IBM SPSS Statistics, IBM 
Corporation, Armonk, NY). As 
previously described,'”** multiple 
imputations using the Markov chain 
Monte Carlo method were performed 
to impute missing data, and 
subsequent analyses were performed 
by using pooled imputed data. The 
maximum missing data that were 
imputed for any variable was 2.2% 
(setting and race and/or ethnicity). P 
<.05 was considered significant. 
First, any available narratives and 
free-text variables were reviewed to 
determine if the sitting device was 
being used as directed at the time of 
death. CSS use guidelines stipulate 
that they be used for transport in 

a vehicle (either parked or in 
motion).'* However, CSSs are now 
being marketed to also be used for 
transport other than in a vehicle (eg, 
stroller and shopping cart). Hazard 
patterns noted in case series of 
injuries and deaths associated with 
CSSs”® suggest that many injuries 
and deaths are associated with 
improper buckling of CSS straps. 
Thus, we defined “as directed” use of 
a CSS as the infant being strapped 
into an appropriately sized CSS while 
the infant was being transported in 
a vehicle (either parked or in motion.) 
For deaths in sitting devices, we 
determined the number of risk 
factors present at the time of death’? 
and categorized them as follows: (1) 
infant risk factors (estimated 
gestation age <37 weeks; birth 
weight <2500 g; intrauterine 
exposure to illicit drugs, heavy 
alcohol use, or misuse of over-the- 
counter medications; infant disability 
or chronic health condition; and 
maternal pregnancy complications), 
(2) supervision risk factors 
(supervisor impaired because of 
alcohol or illicit drugs, supervisor 
with a history of drug or alcohol 
abuse, or supervisor absent, asleep, 
or distracted), and (3) sleep- 
environment risk factors (new sleep 


environment, item in the sleep 
environment, or exposure to 
secondhand smoke). Bed-sharing and 
sleep position were not included as risk 
factors given their relative rarity in or 
irrelevance to deaths occurring in sitting 
devices. Narratives available for deaths 
in CSSs (n = 24) were also examined to 
identify hazards not included as 
variables in the data set. Those hazards 
were described separately for deaths in 
which the CSS was and was not used as 
directed. Second, using the x test, we 
compared risk factors by sleep location 
(in a sitting device versus not in a sitting 
device) and by whether the CSS was 
used as directed. Multivariable logistic 
regression, adjusting for age, race and/ 
or ethnicity, and sex, was used to 
calculate the adjusted odds ratios 
(aORs) of the cause of death, primary 
supervisor, bed-sharing, and any objects 
being present in the sleep environment. 


RESULTS 


Characteristics of Deaths in Sitting 
Devices 


Of 11 779 sleep-related infant deaths 
that were included in the analysis, 348 
(3.0%) occurred in sitting devices 
(Table 1). The majority of deaths in 
sitting devices occurred in CSSs (219; 
62.9%), followed by bouncers, swings, 
and other (122; 35.1%) and strollers 
(7; 2.0%). The median age of infants 
dying in sitting devices was 2.0 months 
(interquartile range 1.0-4.0 months). 
The sitting device was not used as 
directed in nearly two-thirds of deaths 
(62.9%). Among the 219 deaths in 
CSSs, <10% (n = 20) occurred in the 
context of the CSS being used as 
directed (ie, the infant being strapped 
into an appropriately sized CSS while 
the infant was being transported in 

a vehicle, either parked or in motion) 
or when it was unclear if the CSS was 
being used as directed. Among all 
deaths in any sitting device, more than 
three-quarters (81.9%) had a least 1 
risk factor, and more than half (54.9%) 
had 2 or more risk factors. Among 
deaths in any sitting device in which 


the device was used as directed or 
when it was unclear whether it was 
being used as directed (n = 131), the 
most common risk factors were the 
supervisor being asleep (n = 44; 
33.6%), maternal pregnancy 
complications (n = 32; 24.4%), infant 
exposure to secondhand smoke (n = 
30; 22.9%), and having an estimated 
gestational age <37 weeks (n = 30; 
22.9%). Other risk factors that were 
present in >15% of deaths in which 
the sitting device was being used as 
directed or when it was unclear 
whether it was being used as directed 
include the following: item present in 
the sleep environment (n = 26; 19.8%), 
new sleep environment (n = 23; 
17.6%); and low birth weight (n = 21; 
16.0%). A risk factor was present in 
approximately half of each of the 3 
domains: infant risk factors (46.6%), 
supervision risk factors (48.9%), and 
sleep-environment risk factor (48.1%). 
Half of the deaths (50.4%) had risk 
factors in at least 2 of the 3 domains. 


We also examined risk factors associated 
with the subset of deaths occurring in 
CSSs when they were used as directed 
and compared those to risk factors in 
deaths when the CSS was not used as 
directed. Although the comparison was 
likely underpowered, the only significant 
difference was for a supervisor being 
asleep at the time of death (CSS used as 
directed 6.3% versus CSS not used as 
directed 32.0%; P = .044; Appendix 
Table). This association was no longer 
significant in logistic regressions 
adjusted for age and sex. 


Among deaths in CSSs, additional 
information was available from 
narratives (of variable length) for 24 
of the deaths, 7 of which occurred 
when the CSS was used as directed 
and 17 occurred when the CSS was 
not used as directed. Among deaths in 
which the CSS was used as directed, 
the entire CSS was covered with 

a blanket in 2 deaths; 1 infant was 
loosely buckled or not fully buckled 
into the CSS and was overheated, and 
1 infant was left alone in a car in the 
CSS for a prolonged period of time. In 
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TABLE 1 Characteristics of the Study Population by Location of Death 


Characteristics Total (N = 11779), n (%) Not a Sitting Device (n = Sitting Device (n = 348), n (%) P 
11431), n (%) 
Age at the time of death, mo 
0-3 8190 (69.3) 7951 (69.6) 259 (68.7) 29 
4—6 2658 (22.4) 2001 (22.3) 87 (25.0) 
7-11 951 (8.1) 929 (8.1) 22 (6.3) 
Infant sex 
Male 6823 (57.9) 6623 (57.9) 200 (57.5) 84 
Female 4956 (42.1) 4808 (42.1) 148 (42.5) 
Race and/or ethnicity of infant 
Hispanic 2052 (17.3) 1968 (17.2) 64 (18.4) 63 
Non-Hispanic white 5272 (44.8) 0112 (44.7) 160 (46.0) 
Non-Hispanic African American 5755 (31.7) 5626 (31.7) 107 (30.8) 
Other 742 (6.3) 725 (6.3) 17 (4.9) 
Cause of death 
SUID or SIDS 5278 (27.8) 3151 (27.6) 127 (36.5) <.0001 
ASSB 3407 (28.9) 3349 (29.3) 58 (16.7) 
Unknown or undetermined 9094 (43.3) 4931 (43.1) 163 (46.8) 
Primary caregiver at time of death 
Parent or guardian 11458 (97.3) 11118 (97.3) 340 (97.7) 80 
Relative or friend 182 (1.6) 179 (1.6) 
Foster parent 102 (0.9) 99 (0.9) Í 
Other 37 (0.3) 35 (0.3) i 
Primary supervisor at time of death 
Parent or guardian 9004 (76.5) 8743 (76.5) 261 (75.0) 0007 
Relative 662 (5.6) 649 (5.7) 13 (3.7) 
Friend 85 (0.7) 82 (0.7) a 
Baby-sitter 598 (3.4) 575 (5.5) 25 (6.6) 
Child care provider 157 (1.5) 145 (1.5) 12 (5.5) 
Other 45 (0.4) 44 (0.4) a 
Unknown 1425 (12.1) 1590 (12.2) 30 (10.1) 
Setting 
Child’s home 9444 (80.2) 9221 (80.7) 225 (64.1) <.0001 
Relative’s or friend’s home 1423 (12.1) 1365 (11.9) 08 (16.7) 
Foster home 123 (1.0) 121 (1.1) 
Child care center or baby-sitter’s O11 (4.3) 480 (4.2) 31 (8.9) 
home 
Motel, RV, trailer, or park 112 (1.0) 103 (0.9) 9 (2.6) 
Vehicle in motion or temporarily 20 (0.2) j 19 (5.5) 
parked 
Shopping or errands (not in 8 (0.1) a s 
vehicle), restaurant, or on walk 
in stroller 
Other 138 (1.2) 136 (1.2) i 
Bed-sharing <.0001 
Yes 6604 (56.1) 6589 (57.6) 19 (4.3) 
No 5175 (43.9) 4842 (42.4) 333 (95.7) 
=1 object in sleep environment <.0001 
Yes 5954 (33.6) 3883 (34.0) 71 (20.4) 
No 7825 (66.4) 7048 (66.0) 277 (79.6) 
RV, recreational vehicle. 
a Frequency suppressed because cell size <6. 
the 3 other deaths in which the CSS was occurred when the infant was loosely these deaths, the infant had a risk factor 


used as directed, there were no obvious buckled or not fully buckled into the present: supervisor asleep (n = 3); 
hazards noted, although 1 infant had CSS, 2 occurred while the entire CSS was intrauterine exposure to illicit drugs, 
a history of extreme prematurity covered by a blanket, and in 1 death, the heavy alcohol use, or misuse of over- 
(<28 weeks estimated gestational age) CSS had shifted position in a dangerous the-counter medications (n = 2); 

and extremely low birth weight (<1000 manner. In 8 deaths in which the CSS prematurity (n = 1); and an item 

g). Among the deaths in which the CSS was not used as directed, there were no present in the sleep environment 
was not used as directed, 4 deaths obvious hazards, although in all but 3 of (n = 1). 
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Comparison of Deaths in Sitting 
Devices to Deaths Not in Sitting 
Devices 


A higher proportion of infants who died 
in sitting devices had SUID or SIDS or an 
unknown or undetermined cause of 
death compared with infants who did 
not die in sitting devices (P < .0001; 
Table 1). The majority of deaths 
occurring both in sitting devices and not 
in sitting devices occurred in the child’s 
home. Nearly two-thirds (64.1%) of 
deaths in sitting devices, including 74.3% 
of deaths in strollers and more than half 
(51.6%) of deaths in car seats, occurred 
in the child's home. Among deaths in 
sitting devices, higher percentages of 
deaths in CSSs occurred at a relative’s or 
friend’s home (21.9%) and at a child care 
center or baby-sitter’s home (12.3%) 
than deaths in strollers and bouncers or 
swings (total percentage at both 
locations 11.5%; P < .0001). 


After adjusting for age, sex, and race 
and/or ethnicity and in comparison 
with deaths that did not occur in 

a sitting device, deaths in sitting devices 
had higher odds of occurring under 
nonparental supervision, such as baby- 
sitters (aOR 2.0; 95% confidence 
interval [CI] 1.3-3.2) and child care 
providers (aOR 2.8; 95% CI 1.5-5.2), 
than deaths occurring under parental 
supervision (Table 2). Relative to infants 
who died while not in sitting devices, 
infants who died in a sitting device were 
half as likely to have an object in the 
sleep environment (aOR 0.5; 95% CI 
0.4-0.6) or to have a cause of death of 
ASSB (aOR 0.4; 95% CI 0.3-0.6; 
reference = SIDS and SUID). 


DISCUSSION 


In this large data set of 11 779 infant 
sleep-related deaths, 3% occurred in 
a sitting device, consistent with other 
published data.°*° The majority of 
deaths in sitting devices were in CSSs. 
The overwhelming majority of infants 
who died in sitting devices died at 
home and under the supervision of 
their parent or guardian. Because 
there are no data available on the 


TABLE 2 Multivariable Analysis of Risk Factors of Infant Deaths in Sitting Devices Compared With 


Deaths Not Occurring in a Sitting Device 


Characteristics 


Primary supervisor at the time of death 
Relative 
Friend 
Baby-sitter 
Child care provider 
Other 
Unknown 
Parent or guardian 
Cause of death 
ASSB 
Unknown or undetermined 
SUID or SIDS 
Bed-sharing 
Bed-sharing 
Not bed-sharing 
=1 object in sleep environment 
Item present 
No item present 


—, not applicable. 


Odds Ratio (95% CI)? P 
0.7 (0.4—1.2) 16 
1.1 (0.4-3.7) 83 
2.0 (1.5—5.2) 002 
2.8 (1.5—5.2) 002 
0.7 (0.1—5.2) 1D 
0.8 (0.6-1.2) óD 

Reference = 
0.4 (0.3—0.6) <.001 
0.8 (0.7—1.1) 14 

Reference = 

0.03 (0.02—0.05) <.001 

Reference = 
0.5 (0.40.6) <.001 

Reference = 


a Adjusted by age category, sex, and race and/or ethnicity. Reference is death not occurring in a sitting device. 


relative amount of time infants spend 
sleeping in cribs, bassinets, sitting 
devices, and other locations, this 
study cannot compare the relative 
risk of sleeping in a sitting device 
compared with other locations. 
However, these results do serve as 

a reminder that sleep-related infant 
deaths do occur in sitting devices, 
including CSSs. 


CSSs are a safe and effective way of 
transporting an infant and should 
always be used when transporting an 
infant in a motor vehicle, whether the 
infant is awake or asleep. The AAP 
notes that CSSs are designed as safety 
devices to be used in motor vehicles 
and not as general infant sitting 
devices’* and advises that they be 
used only for travel.’**” If a child falls 
asleep in a CSS while actively 
traveling in a motor vehicle, they 
should remain in the CSS until they 
are no longer traveling. If they are 
still asleep when they are no longer 
traveling, they should then be placed 
in a crib or bassinet. Thus, 
theoretically, few deaths in sitting 
devices should occur outside of 
traveling contexts. However, our 
findings indicate that in the great 
majority of deaths in sitting devices, 


the infant was not being transported. 
Although parents and other 
caregivers may put an infant in 

a sitting device with the intent of 
Sleep, the AAP recommends against 
this because of safety concerns. 
Although we cannot ascertain from 
this study all of the circumstances 
surrounding the infant at the time of 
death, we believe that it is prudent 
for parents and caregivers to remove 
the infant from the sitting device 
when they fall asleep and place them 
in a bassinet or crib when they are 
not transporting the infant in a motor 
vehicle. After leaving the motor 
vehicle, infants should be moved to 
a flat, firm surface when it is practical 
to do so.’ At a minimum, infants 
should not sleep in these devices 
unsupervised. Although this data set 
did not provide detail regarding 
buckling, a previous study found that 
several infants who died in sitting 
devices had not being properly 
buckled,’ and we agree that infants 
should also always be buckled with 
a 5-strap harness, even when asleep, 
to reduce the risk of strangulation 
and positional asphyxia. Pediatricians 
should encourage parents and 
caregivers to include a portable crib 
or bassinet when traveling for use 
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once they reach their destination if it 
is likely that their infant will be 
asleep. 


Despite AAP recommendations, 
infants often spend prolonged periods 
of time in CSSs when not traveling, 
and the CSS frequently is used as an 
alternative to a crib or bassinet.° © 
This may in part be because CSSs are 
ubiquitous and because they are 
convenient for parents and caregivers 
to use. Indeed, our analysis revealed 
that nearly two-thirds of deaths in 
sitting devices occurred in the context 
of a sitting device that was not being 
used as directed or for its intended 
purpose. This is consistent with US 
Consumer Product Safety Commission 
injury data that estimate that >8000 
infants are evaluated each year in 
hospital emergency departments for 
injuries associated with CSSs that are 
sustained outside of a motor vehicle.® 


It is likely that caregivers find CSSs and 
other sitting devices to be a convenient 
means of temporarily holding a child 
while the caregiver is busy with other 
tasks. Caregiver confidence in the 
safety of CSS also is likely an important 
driver of the behavior. Knowing that 
CSSs are for safety and are “approved,” 
some caregivers may also believe that 
the use of CSSs for purposes outside of 
travel are also safe. Additionally, 
financial constraints may play a role in 
the inappropriate use or overuse of 
sitting devices for sleeping purposes. 
Although this study did not look at the 
motive behind placing an infant in 

a CSS for nontransportation reasons, it 
is worth considering that some 
caregivers might have been unable to 
afford a portable crib or other flat 
sleeping surface and used a CSS 
instead. 


Although the majority of deaths in 
sitting devices were at home and 
under the supervision of a parent or 
guardian, when compared with other 
deaths, infants dying in sitting devices 
had higher odds of having a child care 
provider or baby-sitter as the primary 
Supervisor at the time of death 


(compared with a parent or guardian). 
Nonparental caregivers should be 
advised not to keep infants in sitting 
devices for sleep. Additionally, 
environments such as a relative’s or 
friend's home may not be well 
equipped to safely accommodate the 
infant during sleep. In such cases, the 
use of a portable crib or bassinet 
should be arranged. Recent campaigns 
to educate child care providers on safe 
sleep-environment practices have 
been successful in improving 
knowledge and safe sleep practices.*® 
However, these findings suggest that 
there are still gaps in the public 
knowledge regarding the appropriate 
usage of sitting devices for infant care 
and that all caregivers, including 
nonparental caregivers, must be 
educated in this regard. 


This study has several limitations. 
Despite the NCFRP being a large case- 
reporting system, not every state 
participates, and only infant deaths 
reviewed by infant and child fatality 
teams are generally reported. 
Therefore, the NCFRP may not 
include all infant deaths in the 
participating states. Without 
denominator data, it is not possible to 
compare the collected data with vital 
Statistics. As a result, it is not possible 
to determine the relative risk of infant 
sleep in a sitting device compared 
with a crib or bassinet. Additionally, 
some data items were collected from 
caregiver interviews, and thus, recall 
bias and missing data are limitations. 
Furthermore, the quality of data is 
dependent on the reporter’s 
experience and personal 
interpretation, and there may have 
been variability in how data were 
reported. In our examination of 
deaths in sitting devices, it is possible 
that some supervisors whom we 
deemed as being impaired because of 
distraction, illness, or disability were 
still capable of supervising the infant; 
however, these deaths comprised 

a small minority of the deaths in 
sitting devices. In addition, not all risk 
factors included in our analysis of 


deaths in sitting devices (eg, maternal 
pregnancy complications) have been 
definitely linked to a heightened risk 
of infant sleep-related mortality. 
Finally, the absence of narratives and 
character limits on the narratives and 
other free-text variables posed 
another challenge in fully 
understanding the context in which 
an infant died in a sitting device. 


CONCLUSIONS 


Deaths in sitting devices comprise 
~3% of all infant sleep-related deaths, 
with the majority of these deaths 
occurring in CSSs. The majority of 
sitting-device-related infant deaths 
occur in nontraveling situations. 
Although our study does not examine 
why these factors are associated with 
infant death in sitting devices, our 
findings underline the significance of 
educating caregivers about the 
importance of using sitting devices 
only for their intended purpose, such 
as using CSSs only for transportation. 
Sitting devices are not an acceptable 
substitute for a crib or bassinet. Using 
CSSs for sleep in nontraveling contexts 
may pose a risk to the infant. 
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BACKGROUND AND OBJECTIVES: Use of commercial direct-to-consumer (DTC) telemedicine outside of 
the pediatric medical home is increasing among children, and acute respiratory infections 
(ARIs) are the most commonly diagnosed condition at DTC telemedicine visits. Our objective 
was to compare the quality of antibiotic prescribing for ARIs among children across 3 settings: 
DTC telemedicine, urgent care, and the primary care provider (PCP) office. 


METHODS: In a retrospective cohort study using 2015-2016 claims data from a large national 
commercial health plan, we identified ARI visits by children (0-17 years old), excluding visits 
with comorbidities that could affect antibiotic decisions. Visits were matched on age, sex, 
chronic medical complexity, state, rurality, health plan type, and ARI diagnosis category. 
Within the matched sample, we compared the percentage of ARI visits with any antibiotic 
prescribing and the percentage of ARI visits with guideline-concordant antibiotic 
management. 


RESULTS: There were 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits for 
ARIs in the matched sample. Antibiotic prescribing was higher for DTC telemedicine visits 
than for other settings (52% of DTC telemedicine visits versus 42% urgent care and 31% PCP 
visits; P < .001 for both comparisons). Guideline-concordant antibiotic management was 
lower at DTC telemedicine visits than at other settings (59% of DTC telemedicine visits versus 
67% urgent care and 78% PCP visits; P < .001 for both comparisons). 


CONCLUSIONS: At DTC telemedicine visits, children with ARIs were more likely to receive 
antibiotics and less likely to receive guideline-concordant antibiotic management compared to 


children at PCP visits and urgent care visits. 
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WHAT’S KNOWN ON THIS SUBJECT: Authors of 
previous studies identified concerns about antibiotic 
prescribing for adults using direct-to-consumer (DTC) 
telemedicine. No study has examined antibiotic 
prescribing during pediatric DIC telemedicine visits. 


WHAT THIS STUDY ADDS: Compared with urgent care 
and primary care visits, children receiving care for 
acute respiratory infections during DIC telemedicine 
visits are more likely to receive antibiotics and less 
likely to receive guideline-concordant antibiotic 
management. 


To cite: Ray KN, Shi Z, Gidengil CA, et al. Antibiotic Prescribing 
During Pediatric Direct-to-Consumer Telemedicine Visits. 
Pediatrics. 2019;143(5):e20182491 
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Many commercial companies offer 
direct-to-consumer (DTC) 
telemedicine visits in which patients 
are connected to physicians outside 
of the medical home through audio- 
only or audio-video conferencing on 
their personal devices.’ DTC 
telemedicine companies highlight the 
convenience of this care and report 
high patient satisfaction.” * Access to 
telemedicine has expanded, with 96% 
of large business insurance plans now 
offering coverage for employees and 
their families.” As a result, millions of 
children now have access to DTC 
telemedicine, and use is rapidly 
increasing.° 


The American Academy of Pediatrics 
discourages the use of DTC 
telemedicine outside of the medical 
home for acute pediatric care, citing 
concerns such as limited physical 
examination capabilities, lack of 
ongoing patient-provider 
relationships, and lack of access to 
patient records, all of which may lead 
to lower-quality care.’ The American 
Academy of Pediatrics further notes 
that these issues may be particularly 
important with children who may 
have difficulty verbalizing their 
symptoms.’ The American 
Telemedicine Association echoed 
many of these concerns,” adding that 
the use of personal devices may also 
impact technical quality and privacy 
of care. The American Telemedicine 
Association further suggests that DTC 
telemedicine outside of the medical 
home should not be used for children 
<2 years old. Both organizations 
emphasize the need for telemedicine 
visits to adhere to the same standards 
expected of in-person care, including 
adherence to professional guidelines 
for appropriate examination, 
diagnostic testing, and treatment.’® 


To date, there is limited evidence 
regarding the quality of care during 
pediatric DTC telemedicine visits. 
Early studies of adult patients using 
DTC telemedicine compared with 


physician office visits have identified 
more inappropriate antibiotic 
prescriptions for bronchitis’ and 
more broad-spectrum antibiotic 
use.” The quality of DTC 
telemedicine care for children may 
differ for several reasons. First, there 
is heightened reliance on the physical 
examination among children who 
cannot fully articulate symptoms.’ 
Second, care requires knowledge of 
pediatric-specific guidelines.**~*” 
Third, rates of guideline-concordant 
antibiotic management at physician 
visits is higher for children than for 
adults**; therefore, the relative 
difference in quality of care at DTC 
telemedicine visits may be different. 


To fill this gap in knowledge about the 
quality of commercial DTC 
telemedicine care for children, we used 
data from a large national health plan 
to compare the quality of antibiotic 
prescribing for acute respiratory 
infections (ARIs) at pediatric DTC 
telemedicine, urgent care, and primary 
care provider (PCP) visits. We focus on 
ARIs because authors of previous 
studies identified concerns about 
antibiotic prescribing during pediatric 
ARI visits in other settings,'*"*? and 
because ARIs account for the majority 
of DTC telemedicine visits.° 


METHODS 


Study Population 


We performed a retrospective cohort 
study using 2015-2016 claims data 
from a large national commercial 
health plan, which provides coverage 
for roughly 4 million children 
annually from all 9 census divisions 
and contracts with a single national 
DTC telemedicine vendor. Children 
age 0 to 17 years were included, with 
no minimum enrollment duration 
requirement. We excluded children 
who lacked pharmaceutical coverage 
during the month of their visit to 
ensure record of pharmacy claims. 


identifying ARI Visits 


ARI visits were identified by using 
previously published methods on the 
basis of International Classification 
of Diseases, Ninth Revision, Clinical 
Modification (ICD-9-CM) codes.’??%" 
Consistent with previous 
work,!®1>2021 ARI diagnoses 
included diagnoses for which 
antibiotics may be warranted 
(sinusitis [461.x], pneumonia [481. 
x-483.x, 485.x-486.x], streptococcal 
pharyngitis [034.x], acute otitis media 
[382.x]) and diagnoses for which 
antibiotics are not warranted (eg, viral 
upper respiratory infection [URI], 
bronchiolitis, viral pharyngitis, serous 
otitis media [381.x, 460.x, 462-466.x, 
480.x, 490.x]). We excluded bacterial 
pneumonia visits because of the small 
number of DTC telemedicine visits 
identified (n = 11). We converted 
International Classification of Diseases, 
10th Revision, Clinical Modification 
(ICD-10-CM) codes to ICD-9-CM codes 
using a modified crosswalk created by 
the Centers for Medicare and Medicaid 
Services (see Supplemental Table 5). 


To examine the first ARI visit within 
an episode of illness, we identified 
“index” ARI visits with no ARI visits at 
any location in the previous 21 days, 
which is consistent with previous 
studies.°** Subsequent ARI visits 
within 21 days of an index visit were 
considered follow-up care. We 
excluded episodes as detailed in 
Supplemental Fig 2. Using 

a previously published methodology, 
we excluded episodes for which the 
index visit included a diagnosis 
indicating chronic conditions (eg, 
history of transplant) or additional 
codiagnoses (eg, urinary tract 
infection) that could impact antibiotic 
prescribing decisions (n = 60 947; 
4.7%) (Supplemental Table 6).'*7* 
These exclusion criteria were 
originally developed for adult 
populations. Although not all of these 
exclusion criteria are reasons to 
deviate from pediatric ARI guidelines, 


Downloaded from www.aappublications.org/news by guest on April 8, 2019 


RAY et al 


we applied them to study a set of 
visits consistent with other studies. 


Identifying Setting of Care 


ARI episodes were categorized as 
DTC telemedicine, urgent care, or PCP 
on the basis of the index visit setting. 
We identified setting using place of 
Service codes, Current Procedural 
Terminology codes, insurer provider 
specialty codes, and organizational 
national provider identifiers and tax 
identification numbers for urgent care 
and DTC telemedicine companies 
through previously described methods 
(additional details in Supplemental 
Information).** ARI episodes that 
began with index visits to settings 
other than DTC telemedicine, urgent 
care, or PCP offices (eg, emergency 
department, retail-based clinics) were 
not included. 


Outcomes 


Our 2 primary outcomes, compared 
across settings, were (1) the 
proportion of visits that resulted in an 
antibiotic prescription and (2) the 
proportion of visits that resulted in 
antibiotic management that was 
concordant with guidelines for the 
visit diagnosis. 


We identified all filled prescriptions 
on the day of the index visit and 
subsequent 2 days and assigned these 
to the index visit, consistent with 
previous methods.**®? We included 
all systemic antibiotics (ie, oral, 
intramuscular) and excluded topical, 
ophthalmic, or otic antibiotics. We 
determined guideline concordance of 
antibiotics with visit diagnosis, 
assigning each visit to 1 of 5 categories 
(Supplemental Table 7).*°~° 
“Antibiotic prescribed” visits included 
visits categorized as guideline- 
concordant antibiotic use, 
guideline-non-concordant antibiotic 
use, and unnecessary antibiotic use. 
“Guideline-concordant antibiotic 
management” included visits 
categorized as guideline-concordant 
antibiotic use for sinusitis, otitis 
media, or streptococcal pharyngitis (ie, 


amoxicillin or amoxicillin-clavulanic 
acid for sinusitis," amoxicillin or 
amoxicillin-clavulanic acid for otitis 
media,!! and penicillin or amoxicillin 
for streptococcal pharyngitis*’), no 
antibiotic use for sinusitis or otitis 
media (also guideline-concordant 
management for these diagnoses’ ”*”), 
or guideline-concordant nonuse of 
antibiotics for diagnoses that do not 
warrant antibiotics (eg, 
bronchiolitis’). 


As secondary outcomes, we compared 
the occurrence of streptococcal 
testing within 1 day for children 
diagnosed with streptococcal 
pharyngitis?” as well as return visits 
within 2 and 21 days of an index visit 
to any setting (eg, PCP, DTC 
telemedicine, urgent care, retail clinic, 
emergency department). We did not 
have streptococcal testing results. 


Patient and Visit Characteristics 


Patient age, sex, and high-deductible 
health plan (HDHP) status were 
determined from insurance 
enrollment files. Using the patients’ 
Zip code, we determined census 
region and rural-urban commuting 
area codes.*° We determined patient 
chronic medical complexity using the 
Pediatric Medical Complexity 
Algorithm Version 2 applied to 
2014-2016 inpatient and outpatient 
claims.*” This algorithm is used to 
identify children with 3 levels of 
chronic medical complexity: complex 
chronic disease, noncomplex chronic 
disease, and no chronic disease.” 


Matching 


To address differences in patients and 
conditions managed in the 3 settings, 
we used coarsened-exact matching,”” 
which prunes observations to achieve 
covariate balance between groups. 
Instead of matching to a fixed number 
of visits, coarsened-exact matching 
matches DTC telemedicine visits to 
many matched urgent care visits and 
PCP visits and then weights each 
stratum or matched set.*° Specifically, 
we matched DTC telemedicine visits 


to urgent care visits and separately to 
PCP visits on the basis of age (in 
years), sex, chronic medical 
complexity, state, rural or urban area, 
HDHP status, and diagnosis category 
(eg, otitis media, viral URI). 


Statistical Analysis 


Across ARI visits for the 3 settings of 
care, we described patient 
characteristics and diagnoses. In the 
unmatched sample and the matched 
weighted sample, we examined 
antibiotic prescribing and guideline- 
concordant antibiotic management 
for DTC telemedicine visits compared 
with urgent care and compared with 
PCP offices using x^ tests. We include 
both unmatched and matched results 
because of potential subjectivity in 
diagnoses, which may then drive 
variability in antibiotic prescribing.*® In 
the matched weighted sample, we also 
compared streptococcal testing and 
return visits by setting using x’ tests. 
We adjusted for multiple comparisons 
in our main results using the 
Benjamini-Hochberg false discovery 
rate,’ yielding a P value of .0342. 


To examine whether differences in 
antibiotic prescribing varied by 
patient characteristics, we performed 
further analysis that was focused on 2 
variables that may impact guideline 
adherence: child age and chronic 
medical conditions. Within each 
category of child age and medical 
complexity, we examined antibiotic 
testing and guideline concordance for 
DTC telemedicine compared with 
urgent care and PCP offices using x” 
tests and then tested the significance 
of an interaction term between age or 
medical complexity and visit setting 
using logistic regression. 


We performed 2 sensitivity analyses 
to examine the robustness of our 
findings to specific assumptions. 
First, we conducted a sensitivity 
analysis in which we redefined 
guideline-concordant antibiotic 
management so that it no longer 
included “no antibiotic use” for 
sinusitis and otitis. Because 
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guidelines allow for observation 
without antibiotic use in specific 
circumstances for sinusitis and otitis 
media.’*’'* we considered no 
antibiotic use to be a guideline- 
concordant management option for 
these diagnoses in our main analysis 
but not in this sensitivity analysis. 
Second, we examined antibiotic 
prescribing and guideline-concordant 
management after excluding ARI 
visits with a follow-up visit within 

2 days because visits with such 
early follow-up might result in 
miscategorization of the site of 
antibiotic prescribing. 


Analyses were conducted in SAS 
version 9.4 (SAS Institute, Inc, Cary, 
NC). Deidentified data were used in 
this analysis, which was determined 
to be exempt by Harvard Medical 
School's Institutional Review Board. 


RESULTS 


In the unmatched sample, there were 
5431 index ARI visits to DTC 
telemedicine, 87555 to urgent care, 
and 1132116 to PCPs in 2015-2016 
that met inclusion criteria. Compared 
with urgent care and PCP visits, DTC 
telemedicine visits were more likely 
to involve children who were older, 
enrolled in HDHPs, and in the South 
(Table 1). Among DTC telemedicine 
visits, 5% were for children <2 years 
old. At DTC telemedicine visits, 
children were more likely to receive 
a diagnosis of viral ARI (68% vs 54% 
urgent care; 59% PCP visits) or 
sinusitis (18% vs 8% urgent care; 
10% PCP visits) and less likely to 
receive a diagnosis of otitis media 
(8% vs 26% urgent care; 23% PCP). 


In the unmatched cohort (Table 2), 
DTC telemedicine visits were 
associated with substantially 
increased antibiotic prescribing (52% 
vs 46% urgent care; 33% PCP; P < 
.001 for both) and decreased 
guideline-concordant antibiotic 
management compared with other 
settings (60% vs 71% urgent care; 
80% PCP; P < .001 for both). 


Antibiotic Prescribing in Matched 
Visits 

After matching, the weighted sample 
included 4604 DTC telemedicine, 

38 408 urgent care, and 485 201 PCP 
visits with improved balance on 
observed characteristics, including 
child age, geographic region, and visit 
diagnosis (Table 1, Supplemental 
Table 8). 


At DTC telemedicine visits, children 
were more likely to receive an 
antibiotic compared with matched 
visits at other settings (52% vs 42% 
urgent care; 31% PCP; P < .001 for 
both comparisons; Table 2). At DTC 
telemedicine visits, children were less 
likely to receive guideline-concordant 
antibiotic management (59% vs 67% 
urgent care; 78% PCP; P < .001 for 
both comparisons). The difference in 
guideline-concordant antibiotic 
management rates was primarily 
driven by antibiotic prescribing for 
visits with viral ARI diagnoses that do 
not warrant antibiotics. For such 
visits, antibiotics were appropriately 
not prescribed in only 54% of DTC 
telemedicine visits compared with 
66% of urgent care visits (P < .001) 
and 80% of PCP visits (P < .001; 
Table 3). 


Streptococcal Testing and Return 
Visits 

In the matched sample, among visits 
with a diagnosis of streptococcal 
pharyngitis, streptococcal testing was 
performed for 4% of DTC 
telemedicine visits compared with 
75% of urgent care and 68% of PCP 
visits (P < .001 for both; Table 4). 
Subsequent visits within 2 days 
occurred for a slightly higher 
percentage of DTC telemedicine visits 
(5% vs 2% urgent care; 1% PCP; P < 
.001 for both). 


Stratified Analysis by Patient 
Characteristics 


Compared with urgent care and PCP 
visits, DTC telemedicine visits had 
significantly increased antibiotic 
prescribing and decreased guideline- 


concordant antibiotic management 
within each category of age and 
chronic medical complexity (Fig 1). 
The 1 exception was that DTC 
telemedicine and urgent care results 
were not statistically different for the 
youngest children (0-1 year old), for 
which the sample size was small. 
Interaction terms indicated that the 
degree of difference between DTC 
telemedicine and PCP visits varied 
significantly with age for antibiotics 
prescribed (interaction term P < 
.001) and guideline concordance 
(interaction term P = .004); other 
interaction terms were not 
significant. 


Sensitivity Analyses 


When no longer classifying “no 
antibiotic use” as guideline- 
concordant for otitis media and 
sinusitis, rates of guideline- 
concordant antibiotic management 
remained lower overall at DTC 
telemedicine (51% vs 59% urgent 
care; 69% PCP; P < .001 for both; 
Supplemental Table 9). Analysis 
excluding index visits with follow-up 
visits within 2 days did not differ 
meaningfully from main results. 


DISCUSSION 


Children treated for ARIs at 
commercial DTC telemedicine visits 
were substantially more likely to 
receive an antibiotic and less likely to 
receive guideline-concordant 
antibiotic management than children 
presenting to urgent care or primary 
care. These quality differences were 
observed in metrics agnostic to visit 
diagnosis (antibiotic prescribing rate) 
and conditional on diagnosis 
(guideline-concordant antibiotic 
management). These differences were 
primarily driven by increased use of 
antibiotics for visits receiving viral 
diagnoses. 


These differences in antibiotic 

prescribing for children contrast with 
previous studies of DTC telemedicine 
quality among adult patients in which 
quality differences have been smaller 
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Standardized 
Difference,” % 


PGP, No. 
(%) 
01113 
(11) 
68 080 
(14) 


After Matching? 
Standardized 
Difference,” % 


Urgent 

Care, 
No. (%) 
5589 (14) 


DIC 


Telemedicine, 
No. (%) 


646 (14) 


Pediatric Visits 


85.9 


Standardized 
Difference,” % 


PGP, No. (%) 
113 592 (10) 


Before Matching 
122.1 


Standardized 
Difference,” % 


Care, 
No. (%) 


Urgent 
7101 (8) 


No. (%) 


DTC Telemedicine, 
966 (18) 


Diagnosis category® 
Sinusitis 


TABLE 1 Continued 
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TABLE 2 Antibiotic Prescribing and Guideline-Concordant Antibiotic Management in Matched Sample, 2015—2016 


<.001 


68 775 (14) 


<.001 


9451 (25) 


1049 (34) 


<.001 
<.001 
<.001 


<.001 105571 (9) 


<.001 


14467 (17) 


1716 (32) 


091 (11) 


Unnecessary antibiotic use 


No antibiotic use 


085 
<.001 


49 401 (10) 


58 
<.001 


5787 (10) 
18 424 (48) 


455 (9) 
1790 (39) 


193 937 (17) 


14.449 (17) 


285 114 


061 941 (50) 


04 


52912 (37) 


1994 (37) 


Guideline-concordant nonuse 


(58) 


, not applicable. 
a P value determined from x? test for DTC telemedicine versus urgent care visits. 


b P value determined from x? test for DTC telemedicine versus PCP visits. 


c Visits with antibiotic prescribing include those categorized as guideline-concordant use, guideline nonconcordant use, and unnecessary antibiotic use. 


d Percentages reported are the proportion of visits in each setting meeting the specified criteria divided by the total number of ARI visits at that setting (indicated by the top row). 


e Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis), and guideline-concordant nonuse of antibiotics. 


using only telephone. These issues 
reduce the data available to the 
treating DTC telemedicine provider, 
which may be of greater concern in 
pediatric care because of the more 
limited ability of children to 
communicate symptoms. Second, 
there may be differential expectations 
for antibiotics among children and 
parents who use DTC telemedicine 
versus in-person care. Although we 
matched on a number of observed 
variables, parental expectations may 
still differ across settings in ways that 
we were unable to address. Third, 
DTC telemedicine visits outside of the 
medical home lack 3 types of 
continuity: informational (lack of 
medical records), relationship (lack of 
ongoing relationship between the 
provider and patient), and clinical 
management (lack of opportunity to 
manage the patient’s care over 
time),°* which has the potential to 
impact quality of care. However, this 
is unlikely to be the only explanation 
given that authors of previous 
evaluations of retail-based clinics, 
which are also a model of in-person 
care with limited continuity, have 
found comparable quality to 
physician office visits.*™*? Fourth, the 
majority of commercial DTC 
telemedicine visits are for adults, 
such that DTC telemedicine providers 
may have variable pediatric training 
or experience, which could impact 
providers’ knowledge of and 
adherence to guideline-based 
pediatric care.* Prior studies 
indicate that only 9.6% of urgent care 
centers employ any pediatricians,”° 
and pediatricians may also be 
underrepresented in DTC 
telemedicine. Together, these issues 
may increase clinical uncertainty 
during pediatric DTC telemedicine 
visits, prompting physicians to 
prescribe antibiotics “just to be 


safe”2738 


We note additional quality concerns 
raised by our analysis beyond 
antibiotic use. First, the American 
Telemedicine Association has 


Downloaded from www.aappublications.org/news by guest on April 8, 2019 


PEDIATRICS Volume 143, number 9, April 2019 


TABLE 3 Antibiotic Prescribing Outcomes by Diagnosis 


Total ARI Visits, M 
Summative categories 


Visits for which antibiotics may be indicated, n 


Any antibiotic use, n (%) 


Guideline-concordant antibiotic management,° n (%) 


Sinusitis visits, n 
Any antibiotic use, n (%) 


Guideline-concordant antibiotic management, n (%) 


Streptococcal pharyngitis visits, n 
Any antibiotic use, n (%) 


Guideline-concordant antibiotic management, n (%) 


Otitis media visits, n 
Any antibiotic use, n (%) 


Guideline-concordant antibiotic management, n (%) 


Visits where antibiotics not indicated, n 


Guideline-concordant antibiotic management, n (%) 


Bronchiolitis visits, n 


Guideline-concordant antibiotic management, n (%) 


Bronchitis visits, n 


Guideline-concordant antibiotic management, n (%) 


Visits for other viral URIs, n 


Guideline-concordant antibiotic management, n (%) 


Visits where antibiotics prescribed, n 


Guideline-concordant antibiotic management, n (%) 


—, not applicable. 


Pediatric Visits 


DTC Telemedicine Urgent Care P PCP P 
4604 38 408 = 485 201 = 
1265 10553 = 133 314 == 

832 (66) 6766 (64) 25 83914 (63) 038 
922 (73) 7364 (70) 023 93 145 (70) 020 
646 5389 — 68 080 — 
410 (63) 3415 (63) 96 43 219 (63) 99 
471 (73) 3837 (71) 36 49611 (73) 98 
250 2086 = 26 347 — 
178 (71) 1403 (67) va 17315 (66) 069 
136 (54) 974 (47) 021 11424 (43) <.001 
369 3078 = 38 888 = 
244 (66) 1948 (63) 28 23 380 (60) 019 
315 (85) 2553 (83) 24 32 109 (83) 16 
3339 97 855 = 351 887 = 
1790 (54) 18424 (66) <.001 283 114 (80) <.001 
51 263 — 4037 = 
31 (61) 182 (69) 25 3197 (79) 001 
363 2686 — 27 308 — 
160 (44) 1114 (41) 344 11270 (41) 28 
2925 24 906 = 320 542 — 
1599 (55) 17 128 (69) <.001 268 647 (84) <.001 
2381 16 198 — 152 687 — 
561 (24) 4260 (26) 005 52776 (35) <.001 


a P value determined from x? test for DIC telemedicine versus urgent care visits. 
b P value determined from x? test for DTC telemedicine versus PCP visits. 
c Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis visits), 


and guideline-concordant nonuse of antibiotics. 


recommended that commercial DTC 
telemedicine should not be used for 
children <2 years old.® We note that 
5% of studied DTC telemedicine visits 
for ARIs were within this age group 
but that gaps in quality existed for 
children of all ages. Second, accurate 
diagnosis of otitis media in children 
requires visualization of the tympanic 
membranes," and accurate diagnosis 
of streptococcal pharyngitis requires 
streptococcal testing.”” Making these 


diagnoses via telemedicine in the 
absence of reliable visualization and 
testing could in itself be considered 
a quality concern. Third, increased 
rates of follow-up visits after DTC 
telemedicine raise questions about 
the effectiveness and system-level 
efficiency of these visits. 


Quality of care for ARIs may be better 
in other models of acute telemedicine 
care. For example, acute telemedicine 


TABLE 4 Streptococcal Testing and Return Visits in Matched Sample, 2015—2016 


Pediatric Visits 


DTC Urgent P PCP P 
Telemedicine Care 
Visits with diagnosis of streptococcal 250 2086 — 26 547 — 
pharyngitis, N 
Streptococcal testing performed, n (%) 9 (4) 1557 (75) <.001 17818 <.001 
(68) 
Total ARI visits, N 4604 58 408 — 485 201 — 
Return ARI visit within 2 d, n (%) 226 (5) 869 (2) <.001 5875 (1) <.001 


Return ARI visit within 21 d, n (%) 


—, not applicable. 


020 (11) 


5719 (10) <.001 45629 (9) <.001 


a P value determined from x’ test for DIC telemedicine versus urgent care visits. 
b P value determined from x’ test for DTC telemedicine versus PCP visits. 


visits in which peripheral devices 
such as tele-stethoscopes or tele- 
otoscopes (such as offered at 
telemedicine kiosks*”) are integrated 
may allow for enhanced examination 
compared with DTC telemedicine. Using 
telemedicine technology to connect 
patients to their PCP’s office would 
allow telemedicine use within the 
context of continuity of information, 
relationships, and care management.’ 
School-based or day care-based models 
of acute telemedicine can be used to 
integrate peripherals and 
telepresenters.*°** Finally, pediatric- 
specific improvement initiatives may 
also result in improved quality.” 
Examining the variation in quality 
across different models of acute 
telemedicine care may inform optimal 
use of technology for acute pediatric 
concerns. 


Although our focus was DTC 
telemedicine, the quality of antibiotic 
prescribing was also lower at urgent 
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FIGURE 1 


Antibiotic prescribing and guideline-concordant antibiotic management for DIC telemedicine compared with urgent care (UC) and PCP visits, stratified by 
child age and chronic medical complexity. A, Antibiotic prescribing associated with DIC telemedicine visits compared with UC and PCP visits, stratified by 
4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DIC 
telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each 
age or medical complexity category, the values for UC and PCP visits differed significantly from DIC telemedicine with 1 exception: DIC telemedicine and 
UC results were not statistically different for the youngest children (0—1 year old). Interaction terms indicated that the degree of difference between DIC 
telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction 
term P = .004); other interaction terms were not significant. 


care visits compared with PCP visits 
(59% vs 67% guideline concordant). 
Urgent care clinics have grown rapidly 
in the United States,** and urgent care 
and DTC telemedicine share some 
characteristics that might impede 
quality, such as limited continuity. 
Across all 3 settings, our findings 
emphasize the need for improvement in 
guideline-concordant antibiotic use and 
ongoing antibiotic stewardship efforts in 
outpatient settings.“ S 


Our analysis had several limitations. 
As an analysis of insurance claims, we 
did not have additional 


sociodemographic or clinical data and 
thus could not account for variables 
such as patient race, severity of 
illness, time constraints, or family 
expectations, which might drive 
differences in prescribing.*®*” Given 
the data available, we could not 
independently confirm the diagnosis 
made. The differences in diagnosis 
mix in the unmatched data could be 
due to true differences in case mix 
(perhaps due to self-selection to 
specific settings on the part of 
parents) or biases in diagnosis 
(perhaps due to differences in 
information available to clinicians or 


differences in desire to select 

a diagnosis that justifies prescription 
of antibiotics). Insurance claims also 
do not have information on patient 
allergies. Although we would not 
expect substantial differences in rates 
of allergies across settings, we may be 
underestimating overall guideline 
concordance in the absence of allergy 
data. Also, because some children 
were enrolled for brief periods, we 
may not be capturing all children 
with higher medical complexity. 
However, we confirmed that 
identified rates of chronic complex 
disease in the full population of 
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pediatric health plan beneficiaries 
(6.4%) were relatively similar to rates 
identified in other population studies 
(6% in previous claims analysis“). 
We used claims data rather than 
electronic health record or provider- 
reported data. Although this means 
we examined antibiotics filled rather 
than prescribed, in-person visit 
antibiotic use appeared consistent 
with previous studies of antibiotic 
use. Specifically, our rates of 
antibiotic use among unmatched PCP 
visits were comparable to previous 
studies for diagnoses for which 
antibiotics may be indicated (59% vs 
50%-80% in previous literature) and 
diagnoses for which antibiotics are 
not indicated (16% vs 20%-21% in 
previous literature).'*2°°° We did 
not include retail-based clinics in our 
analyses. Authors of other studies 
have found that among adults, retail- 
based clinics have higher rates of 


guideline-concordant care relative to 
PCPs.'>? Physician specialty was not 
available for DTC telemedicine visits, 
so we could not examine whether 
pediatric-specific physician training 
explained any quality differences. We 
note also that our analysis is limited 
to a specific health plan and visits by 
its members to their contracted DTC 
telemedicine vendor, and our analysis 
also did not include Medicaid 
beneficiaries. Antibiotic prescribing 
among other DTC telemedicine 
companies, models, and populations 
may differ. Understanding the impact 
of physician training and 
telemedicine models on quality of 
care during DTC telemedicine should 
be a priority in future work. 


CONCLUSIONS 


Children who receive care via DTC 
telemedicine visits were significantly 


more likely to receive antibiotics and 
less likely to receive guideline- 
concordant antibiotic management 
than children who visit either urgent 
care or PCPs. 
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Sleep-Related Infant Suffocation Deaths 
Attributable to Soft Bedding, Overlay, 
and Wedging 


Alexa B. Erck Lambert, MPH,’ Sharyn E. Parks, PhD, MPH,” Carri Cottengim, MA,” Meghan Faulkner, MA,° Fern R. Hauck, MD, MS,° 
Carrie K. Shapiro-Mendoza, PhD, MPH? 


BACKGROUND: Unintentional suffocation is the leading cause of injury death among infants 

<1 year old in the United States, with 82% being attributable to accidental suffocation and 
strangulation in bed. Understanding the circumstances surrounding these deaths may inform 
prevention strategies. 


METHODS: We analyzed data from the population-based Sudden Unexpected Infant Death Case 
Registry from 2011 to 2014. Cases categorized as explained suffocation with unsafe sleep 
factors (suffocation), per the Centers for Disease Control and Prevention’s Sudden Unexpected 
Infant Death Case Registry classification system, were included and assigned a mechanism of 
obstruction, including soft bedding, overlay, or wedging. We calculated frequencies and 
percentages of suffocation deaths by mechanism and selected demographic and sleep- 
environment characteristics. 


RESULTS: Fourteen percent of sudden unexpected infant death cases were classified as 
suffocation; these cases were most frequently attributed to soft bedding (69%), followed by 
overlay (19%) and wedging (12%). Median age at death in months varied by mechanism: 3 for 
soft bedding, 2 for overlay, and 6 for wedging. Soft-bedding deaths occurred most often in 
an adult bed (49%), in a prone position (82%), and with a blanket (or blankets) obstructing 
the airway (34%). Overlay deaths occurred most often in an adult bed (71%), and infants 
were overlaid by the mother (47%). Wedging deaths occurred most often when the infant 
became entrapped between a mattress and a wall (48%). 


CONCLUSIONS: Safe sleep environments can reduce infant suffocation deaths. Increased 
knowledge about the characteristics of suffocation deaths can help inform prevention 
strategies by targeting highest-risk groups. 
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ARTICLE 


Unintentional suffocation is the 
leading cause of injury death among 
infants (<1 year old) in the United 
States. Eighty-two percent of 
unintentional infant suffocation 
deaths are attributed to accidental 
suffocation and strangulation in bed 
(ASSB). According to US death- 
certificate data, rates of infant deaths 
reported as ASSB per 100 000 live 
births have increased nearly fourfold, 
from 6 deaths in 1999 to 23 deaths in 
2015.” This increase is at least partly 
attributed to changing diagnostic 
preferences and improved death 
investigations.” These injury deaths, 
often resulting from hazards in the 
sleep environment, are preventable. 
Historically, surveillance of infant 
sleep-related suffocation deaths has 
relied on underlying cause-of-death 
codes that are assigned on the basis of 
the cause of death from death 
certificates. However, death 
certificates do not systematically 
collect information about the 
circumstances of death. Differentiating 
a death caused by suffocation from 
other causes of sudden unexpected 
infant death (SUID), such as sudden 
infant death syndrome (SIDS), relies 
on detailed information from the 
death-scene investigation and autopsy, 
for which practices are not standard 
across jusridictions.° Furthermore, 
variations are known to exist in 
diagnostic preferences and the levels 
of evidence death certifiers use to 
designate suffocation as a cause of 
death, leading to a diagnostic shift in 
SUID from SIDS to ASSB.7~° 


In 2009, the Centers for Disease 
Control and Prevention (CDC) created 
the Sudden Unexpected Infant Death 
Case Registry to better understand the 
trends and characteristics associated 
with SUID, including sleep-related 
suffocation.° The SUID Case Registry is 
built on existing Child Death Review 
programs and the National Center for 
Fatality Review and Prevention 
protocols and data system.’ The Case 
Registry is a multijurisdictional, 
population-based surveillance system 


that monitors SUID cases using data 
from death-scene investigations, 
autopsies, medical records, and other 
relevant data sources. Approximately 
one-third of US SUID cases are 
captured in the Case Registry. States 
participating in the Case Registry 
receive assistance from the CDC to 
improve data quality, timeliness, and 
case ascertainment. 


In 2014, the CDC developed the Case 
Registry classification system to 
consistently differentiate SUID cases 
into the following groups: no autopsy 
or death-scene investigation, 
incomplete case information, no 
unsafe sleep factors, unsafe sleep 
factors, possible suffocation with 
unsafe sleep factors, and explained 
suffocations with unsafe sleep 
factors. To understand the factors 
contributing to and mechanisms of 
suffocation deaths, the classification 
system requires the review of 
detailed information about specific 
circumstances of each death, 
including the sleep location and 
position when found. This 
information and the resulting 
classification are important to reliably 
monitor the incidence of and trends 
in SUID and SUID subtypes. 
Identifying factors contributing to 
suffocation deaths occurring in 
unsafe sleep environments addresses 
national and international research 
priorities’ and may help inform 
strategies to reduce these deaths. In 
this study, we describe demographic 
and sleep-environment 
characteristics of SUID cases 
classified as explained suffocation 
with unsafe sleep factors 
(suffocation) per the CDC’s SUID Case 
Registry classification system. This 
classification requires a high level of 
evidence of airway obstruction, which 
is described in detail below. 


METHODS 


Using data from the CDC’s SUID Case 
Registry,° we analyzed infant deaths 
that occurred from 2011 through 


2014 (the most recent year for which 
categorized Case Registry data were 
available) among residents of states 
participating in the Case Registry. 
Infants were defined as children 

<1 year old. We limited the study 
population to deaths classified as 
suffocation per the classification 
system.” Data were derived from the 
states participating in the Case 
Registry at the time of this study, 
including Arizona, Colorado, Georgia, 
Louisiana, Michigan, Minnesota, New 
Hampshire, New Jersey, New Mexico, 
and Wisconsin. In addition to existing 
data-use agreements, all states agreed 
to the use of their deidentified 
aggregate data for this study. 


For these analyses, the authors 
classified Case Registry cases using 
the classification system. Cases were 
classified as suffocation when all of 
the following criteria were met.® First, 
the reported cause on the death 
certificate was one of the following: 
unknown; undetermined; SIDS; SUID; 
unintentional sleep-related asphyxia, 
suffocation, or strangulation; 
unspecified suffocation; cardiac or 
respiratory arrest without other well- 
defined causes; or ill-defined causes 
with potentially contributing unsafe 
sleep factors. Intentional homicides 
were excluded. Second, a complete 
death investigation documented 
where and how the infant was found, 
and a comprehensive autopsy 
included (at a minimum) toxicology, 
imaging, and pathology. Third, the 
infant was found unresponsive in an 
unsafe sleep environment with 

a reliable, nonconflicting witness 
account of a full external airway 
obstruction of both the nose and 
mouth or airway obstruction due to 
compression of the neck and/or 
chest. Finally, there were no other 
potentially fatal findings or 
concerning conditions (eg, sepsis or 
congenital birth defect) reported. 


Using the classification system, the 
authors assigned each suffocation 
case 1 or more mechanisms to which 
the airway obstruction was 
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attributed. Mechanisms included soft 
bedding, overlay, wedging, and other.® 
Soft bedding was assigned when an 
infant’s airway was obstructed by an 
adult mattress, blanket, pillow, couch 
cushion, or other soft object in the 
immediate sleep environment. 
Wedging was assigned when an 
infant’s airway was obstructed as the 
result of being trapped or 
compressed between inanimate 
objects (eg, a mattress and wall). 
Overlay was assigned when there was 
a witnessed account of a person on 
top of or against an infant, 
obstructing the infant's airway. Other 
was assigned only when the airway 
obstruction was caused by something 
in the sleep environment other than 
soft bedding, overlay, or wedging, 
such as a plastic bag. Cases could 
have been assigned >1 mechanism. 
For example, an infant whose nose 
and mouth were obstructed by soft 
bedding (soft bedding), and had his 
or her neck or chest compressed by 
a person (overlay). 


We derived study variables from 
individual fields in the National 
Center for Fatality Review and 
Prevention’s National Fatality Review 
Case Reporting System by combining 
information from different fields or 
after a qualitative review of written 
text.’ Selected variables were those 
most pertinent to the study question 
with the most complete information 
recorded. This study includes infants 
from birth to <1 year old. Age is 
reported in completed months. For 
example, 0 months includes infants 
from birth to <1 month old, and 

11 months includes infants from 11 
to <12 months old. Medicaid is 

a state and/or federal health 
insurance program for people with 
low income and is used as a proxy for 
socioeconomic status. Qualitative 
variables include object obstructing 
the airway, person sharing the sleep 
surface with the infant, person who 
overlaid the infant, body part that 
overlaid the infant, airway 
obstruction type, person who 


overlaid was impaired by drugs or 
alcohol near the time he or she went 
to sleep with the infant, and objects 
infant was wedged between. In cases 
in which the object that obstructed 
the airway was an adult mattress, 
there may have been sheets, blankets, 
or other bedding materials between 
the adult mattress and the infant’s 
airway. We calculated frequencies 
and percentages of suffocation by 
mechanism and selected variables, 
including infant demographics and 
sleep-environmental factors at the 
time of death. Per the data-use 
agreement with states, counts of 1 
through 2 are suppressed in an effort 
to protect confidentiality. Analyses 
were conducted by using SAS for 
Windows version 9.3 (SAS Institute, 
Inc, Cary, NC). 


RESULTS 


Of 1812 cases in the Case Registry 
from 2011 through 2014, 250 (14%) 
were Classified as suffocation per the 
classification system; the remaining 
cases were classified as unexplained 
SUID. Of suffocation cases, 219 were 
assigned 1 mechanism, and 31 were 
assigned 2 mechanisms. Cases with 2 
mechanisms were counted in both 
mechanism groups; thus, 281 
mechanisms were assigned to the 250 
cases. Seven cases were assigned the 
mechanism other. The majority of 
these infants were suffocated by 
plastic bags. Because of the small size 
of this group, these cases are not 
included in the analyses by 
mechanism. Of the mechanisms 
assigned, 190 (69%) were soft 
bedding, 51 (19%) were overlay, and 
33 (12%) were wedging (Table 1). 


A majority of infants were boys, born 
at term (=37 weeks’ gestation), non- 
Hispanic white or African American, 
and insured by Medicaid (Table 1). 
The median age at death was 3 
months, and most deaths occurred in 
the infant’s home. The distribution of 
age at death varied by mechanism 
(Fig 1). Among soft-bedding deaths, 


68% occurred from 1 through 

4 months old (median = 3 months 
old). Among overlay deaths, 71% 
occurred during the first 2 months of 
life (median = 2 months), and no 
deaths occurred after 7 months. More 
than half (58%) of wedging deaths 
occurred from 5 through 8 months 
old (median = 6 months old). Of the 
51 suffocation deaths attributed to 
overlay, 25% were born preterm 
(<37 weeks’ gestation) compared 
with 15% among soft bedding and 
12% among wedging. Also, 49% of 
overlay deaths were exposed to 
prenatal maternal smoking compared 
with 36% among soft bedding and 
27% among wedging. 


Soft Bedding 


Suffocation deaths attributed to soft 
bedding varied by sleep place, 
position found, surface-sharing 
Status, and type of object obstructing 
the airway (Table 2). Among soft- 
bedding deaths, 49% occurred in an 
adult bed, and 27% occurred in a crib 
or bassinet (including portable cribs; 
Table 2). Most (92%) were found in 
a nonsupine position. Of the 92 
infants (48%) who were found 
sharing a sleep surface with another 
person, 61 (66%) were sharing with 
1 or both parents, and 24 (26%) were 
sharing with 1 or more siblings (these 
groups are not mutually exclusive, 
nor are data shown in tables). Of the 
soft objects reported as obstructing 
the infants’ airways, 34% were 
blankets, 23% were adult mattresses, 
and 22% were pillows. 


The type of object obstructing the 
airway differed by infant age. Infants 
<4 months old had their airways 
obstructed by pillows or couch or 
recliner cushions approximately 
twice as often as infants 5 to 

11 months old (25% vs 11% for 
pillows; 12% vs 7% for couch or 
recliner cushions). Infants 5 to 

11 months old had their airways 
obstructed by blankets approximately 
twice as often as infants <4 months 
old (55% vs 27%). 


Downloaded from www.aappublications.org/news by guest on April 22, 2019 


PEDIATRICS Volume 143, number 5, May 2019 


TABLE 1 Selected Characteristics of Suffocation Deaths Overall (n = 250) and by Mechanism (n = 274): 2011—2014 


Overall Soft Bedding 
n % n % 

Overall 250 190 69 
Infant sex 

Male 157 50 104 50 

Female 112 49 85 45 

Missing j i a i 
Gestational age at birth, wk 

Preterm, <57 4] 16 28 15 

Term, =57 202 81 156 82 

Missing 7 ő 6 ő 
Infant race and/or ethnicity 

Non-Hispanic white 101 40 79 59 

Non-Hispanic African American 85 54 og 51 

Hispanic 41 16 56 19 

Other, missing, or unknown 25 9 20 11 
Infant insured by Medicaid 

Yes 176 70 130 68 

No 44 18 36 19 

Unknown 50 12 24 15 
Location of death 

Infant’s home 209 84 154 81 

Friend’s or relatives home 24 10 21 11 

Day care 7 ő 6 ő 

Missing or other 10 4 9 9 
Exposure to prenatal maternal smoking 

Yes 89 56 69 56 

No 138 Do 104 50 

Unknown 25 9 17 9 


Overlay Wedging 

n % n % 
5l 19 ah 12 
26 ol 17 52 
25 49 16 48 
a a a a 
15 25 4 12 
óT T5 28 85 
a a a a 
22 45 15 39 
22 43 15 39 
4 8 a a 
3 6 a a 
44 86 19 08 
ó 6 7 21 
4 8 7 21 
45 88 28 85 
ó 6 ó 9 
a a a a 
a a a a 
25 49 9 27 
19 3/ 21 64 
7 14 ő 9 


The overall column includes all 250 suffocation cases; cases assigned 2 mechanisms are included in the total for each of the assigned mechanisms. 


a Per the data-use agreement with states, counts of 1 through 2 are suppressed in an effort to protect confidentiality. 


Overlay surface with >1 person. Among 


Seventy-one percent of overlay overlay deaths, 7 (14%) were 


deaths occurred in an adult bed, and 
51% were found nonsupine (Table 3). 
Of persons who overlaid these 
infants, 47% were the mother, 25% 
were the father, and 22% were 

a sibling. In 41% of the overlay 
deaths, the infant was sharing a sleep 


overlaid by the mother’s breast (data 
not shown in tables). Most overlay 
deaths (71%) were attributed to neck 
or chest compression as opposed to 
obstruction of the nose and mouth 
(22%) or both or unknown (8%). Of 
the 40 overlay deaths in which an 


2 Overlay 


N N eS) 
© Nn an) 


jæ 
Nn 


Percentage 


0 
Soft bedding 


Wedging 
0 l 2 3 4 5 6 7 8 9 10 11 
Age at Death in Months 


0 


FIGURE 1 

Age distribution for each mechanism of suffocation death (2011—2014; n = 274). In an effort to 
protect confidentiality and per the data-use agreement with states, percentages in the 1% to 7% 
range with a numerator of 1 through 2 are all shown on the y-axis at the same level (5%). 


adult overlaid the infant, 23% of the 
adults were reported to have been 
impaired by alcohol or drugs around 
the time they went to sleep with the 
infant. Among the 10 overlay deaths 
that occurred after an adult fell asleep 
while feeding the infant, 7 were 
breastfeeding (14% of all overlay 
deaths). 


Wedging 


Of the 33 suffocation deaths 
attributed to wedging, 45% were 
sharing a sleep surface at the time of 
death, and 73% occurred in an adult 
bed (Table 4). The objects between 
which the infant was wedged were 
most frequently a mattress and wall 
(48%), followed by a mattress and 
bed frame (including headboard; 
27%). Some infants were wedged 
between a crib mattress and 
something else; these cases all 
involved a crib mattress used with 

a broken crib or a crib mattress that 
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TABLE 2 Suffocation Soft-Bedding Deaths by Age at Death and Sleep Circumstances (n = 190): 2011—2014 


Sleep place 
Adult bed 
Crib or bassinet 
Chair or couch 
Other 
Position found 
Prone 
Side 
Supine 
Sharing a sleep surface with another person 
No 
Yes 
Unknown 
Object obstructing airway 
Blanket(s) 
Adult mattress 
Pillow 
Couch or recliner cushion 
Multiple objects 
Bumper pad 
Other 


Groups do not equal 100% because of rounding. 


All Ages 

n % n 
93 49 Ti 
5l 27 4] 
26 14 22 
20 11 12 
159 82 126 
19 10 10 
14 T 10 
95 00 69 
92 48 T9 

ó 2 i 
64 54 40 
44 20 óD 
42 22 óT 
20 11 17 
10 D a 

D ó a 

D 5 D 


a Per the data-use agreement with states, counts of 1 through 2 are suppressed in an effort to protect confidentiality. 


was ill fitting or not used as 

recommended (eg, an infant between 
a crib mattress and an adult mattress 
that were both placed on the floor or 
an infant between a crib mattress and 
the broken rail of a drop-down crib). 


DISCUSSION 


From 2011 to 2014, 250 of 1812 
(14%) Case Registry cases were 
classified as suffocation as defined by 
the classification system.® Suffocation 
deaths were most frequently 
attributed to an airway obstruction 
by soft bedding, followed by overlay 
and wedging. These deaths represent 
a small proportion of all SUID cases, 
but safe sleep environments can 
prevent suffocation deaths. From 
2011 to 2014, the US SUID rate was 
87 deaths per 100 000 live births, 
according to National Center for 
Health Statistics data.’* If the number 
of deaths proportionally equivalent to 
the 14% of Case Registry cases 
classified as suffocation from 2011 
through 2014 had been prevented, 
the US SUID rate would have dropped 
to 75 deaths per 100 000 live births, 


which is below the Healthy People 
2020 goal of 84.111? 


Our study revealed the following 
commonalities and differences in 
characteristics of suffocation deaths 
stratified by mechanism of airway 
obstruction. Nonsupine sleep position 
and sleeping in an adult bed were 
common characteristics among 
deaths attributed to all 3 
mechanisms. Soft-bedding deaths 
most commonly involved an airway 
obstruction by blankets. Among 
overlay deaths, surface sharing with 
the parent(s) was most common, and 
infants were most often overlaid by 
their mothers. Wedging deaths most 
frequently occurred when the infant 
became stuck between an adult 
mattress and a wall. 


Our findings reveal that factors and 
mechanisms of suffocation differ by 
infant age. Younger infants 

(<4 months old) were more often 
suffocated by overlay or soft bedding 
than wedging. Younger infants are 
less likely to get themselves into 

a wedged position because they are 
less mobile and cannot roll over on 


0—4 mo 


5—11 mo 
% n % 
49 22 50 
28 10 25 
15 4 9 
8 8 18 
86 29 69 
7 9 21 
7 4 10 
47 26 59 
51 17 39 
a a a 
27 24 55 
24 9 20 
25 5 11 
12 3 7 
a a a 
a a a 
5 0 0 


their own. Infants who were 
suffocated by overlay were youngest 
and had a higher proportion born 
preterm than infants who were 
suffocated by soft bedding or via 
wedging. 


Among soft-bedding deaths, more 
than half of infants 5 to 11 months 
old had their airways obstructed by 
blankets compared with less than 
one-third of younger infants. Almost 
half of infants 5 to 11 months old 
whose airways were obstructed by 
blankets were entangled in the 
blankets. It is likely that these older, 
more developed infants were mobile 
enough to become entangled in 
blankets but were not yet 
coordinated enough to free 
themselves. Pillows caused the 
airway obstructions twice as often 
among infants <4 months old 
compared with infants 5 to 

11 months old. Younger infants may 
have lacked the mobility and neck 
strength necessary to lift their heads 
to prevent an airway obstruction, 
especially when placed prone or on 
their side on a pillow.” 
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TABLE 3 Suffocation Overlay Deaths by Sleep Circumstances (n = 51): 2011—2014 


Age at Death 0—6 


a 


mo 
n % 

Sleep place 

Adult bed 36 71 

Chair or couch 13 20 

Crib or bassinet i i 

Other i j 
Found position 

Prone 14 27 

Side 12 24 

Supine 20 49 
Person involved with overlay 

Mother 24 47 

Father 13 20 

Sibling 11 22 

Other ő 6 
Airway obstruction mechanism 

Neck or chest compression 56 T1 

Nose and mouth obstruction 11 22 

Unknown or both 4 8 
Adult fell asleep feeding the infant 

No or unknown 4] 80 

Yes 10 20 
Feeding type among adults who fell asleep feeding the infant (n = 10)° 

Breast T 70 

Bottle ő 50 
Person who overlaid impaired by drugs or alcohol (n = 40)° 

Yes 9 25 

No, not specified, or unknown 31 78 


Groups do not equal 100% because of rounding. 
a There were no deaths after 6 mo of age. 


b Per the data-use agreement with states, counts of 1 through 2 are suppressed in an effort to protect confidentiality. 


© Denominator = adults who fell asleep feeding. 
d Denominator = adult or nonsibling overlays. 


The American Academy of Pediatrics 
(AAP) recommends that infants sleep 
supine in a safety-approved crib, 
bassinet, or portable crib with no soft 
objects or bedding in the sleep area. 
The AAP also recommends that 
infants share a room but not 

a sleep surface.’* Following the AAP 


safe-sleep recommendations can 
prevent suffocation deaths. 


Despite differences in case inclusion 
criteria, definitions, and the process 
of assigning cause of death and 
mechanism, some of our findings are 
consistent with earlier studies 


TABLE 4 Suffocation Wedging Deaths by Sleep Circumstances (n = 33): 2011—2014 


Sleep place 
Adult bed 
Crib or bassinet 
Other (including chair or couch) 
Sharing a sleep surface with another person 
No 
Yes 
Unknown 
Objects infant was wedged between 
Mattress and wall 
Mattress and bed frame 
Crib mattress and other 


Other (eg, in a stroller or between ottoman and couch) 


a Per the data-use agreement with states, counts of 1 through 2 are suppressed in an effort to protect confidentiality. 


n % 
24 75 
12 
5 15 
17 52 
15 45 
a a 
16 48 
9 27 
a a 
a a 


examining infant sleep-related 
suffocation deaths.'>*° Among 
suffocation deaths, there was a higher 
proportion of younger infants. 1-17 
Overlay deaths were more common 
among infants 0 to 2 months old, and 
wedging deaths were more common 
among infants 3 to 6 months old.” 
Pillows, mattresses, and blankets or 
comforters were the most common 
objects obstructing infant airways.” As 
in our study, most overlay deaths 
occurred on an adult bed or couc 
and infants were most often overlaid by 
a parent or sibling.’°** Although an 
infrequent occurrence, other studies 
have also reported overlay deaths when 
the infant’s mother fell asleep 
breastfeeding. Wedging deaths most 
commonly occurred between a mattress 
or bed and wall,’ ’® and those that 
occurred in a crib were related to an ill- 
fitting mattress or faulty crib.°°1”"? 


17,18 
h, 


Our analyses were subject to some 
limitations. A high level of detailed 
evidence about the circumstances at 
death was required to be categorized 
as suffocation with the classification 
system. As a result, there may have 
been infants included in the Case 
Registry during the study years who 
were suffocated by hazards in their 
sleep environments, but the death 
investigation or documentation lacked 
the evidence necessary for the case to 
be categorized as suffocation. Like 
earlier SUID studies, information 
about a surface sharer’s drug and 
alcohol impairment was not 
documented consistently.” +”? 
Furthermore, because this was 

a descriptive study of a population of 
infants who died, we lacked 

a comparison group and thus were 
unable to quantify risk associated with 
infant sleep practices (ie, there was no 
control group). Also, the size of the 
study population limited our ability to 
make meaningful observations about 
some of the smaller groups in our 
stratified analyses. 


Despite these limitations, our data 
source and the resulting analyses 
have several strengths. Case Registry 
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states committed to conducting 
population-based SUID surveillance 
and received targeted assistance and 
resources from the CDC to improve 
data quality and completeness.”” 
Earlier studies have used data derived 
from the National Fatality Review Case 
Reporting System without restricting 
cases to those from Case Registry 
states and thus have had less complete 
data.” The strengths of Case 
Registry data address the limitations 
of other available data sources, such as 
death certificates. Studies using only 
death-certificate data lack specifics 
about the sleep environment and are 
limited by inconsistent classification of 
suffocation deaths. Case Registry data 
include detailed information about the 
sleep environment, allowing for 
classification on the basis of strong 
evidence of suffocation by using 
standardized criteria and definitions? 
and the differentiation of airway 
obstruction by mechanism. Finally, 
because complete and comprehensive 
data about the sleep environment 
were required to classify a case as 
suffocation, there were no missing and 


unknown responses for any sleep- 
environment variables. 


CONCLUSIONS 


Since 1999, there has been a dramatic 
increase in the number of US infant 
deaths attributed to suffocation. 
These observed rates may not be fully 
explained by increased incidence and 
are at least partly attributed to 
changing diagnostic preferences and 
improved death investigations.” 
Regardless, unintentional injury 
deaths, such as those categorized as 
suffocation in our study, can be 
prevented by following infant safe- 
sleep practices. The safest place for 
infants to sleep is on their backs, on 
an unshared sleep surface, in a crib or 
bassinet in the caregivers’ room, and 
without soft bedding (eg, blankets, 
pillows, and other soft objects) in 
their sleep area.’* Improving our 
understanding of the characteristics 
and risk factors (eg, age differences 
and sleep-environment 
characteristics) of suffocation deaths 
by mechanism of airway obstruction 


can inform the development of more 
targeted strategies to prevent these 
injuries and deaths. 
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Accuracy of Autism Screening in 
a Large Pediatric Network 
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Marsha Gerdes, PhD,°* Juhi Pandey, PhD,“* Susan E. Levy, MD, MPH,?*® Robert T. Schultz, PhD,*"® Judith S. Miller, PhD% 


BACKGROUND: Universal screening is recommended to reduce the age of diagnosis for autism 
spectrum disorder (ASD). However, there are insufficient data on children who screen 
negative and no study of outcomes from truly universal screening. With this study, we filled 
these gaps by examining the accuracy of universal screening with systematic follow-up 
through 4 to 8 years. 


METHODS: Universal, primary care-based screening was conducted using the Modified Checklist 
for Autism in Toddlers with Follow-Up (M-CHAT/F) and supported by electronic 
administration and integration into electronic health records. All children with a well-child 
visit (1) between 16 and 26 months, (2) at a Children’s Hospital of Philadelphia site after 
universal electronic screening was initiated, and (3) between January 2011 and July 2015 
were included (N = 25999), 


RESULTS: Nearly universal screening was achieved (91%), and ASD prevalence was 2.2%. 
Overall, the M-CHAT/F’s sensitivity was 38.8%, and its positive predictive value (PPV) was 
14.6%. Sensitivity was higher in older toddlers and with repeated screenings, whereas PPV 
was lower in girls. Finally, the M-CHAT/F’s specificity and PPV were lower in children of color 
and those from lower-income households. 


CONCLUSIONS: Universal screening in primary care is possible when supported by electronic 
administration. In this “real-world” cohort that was systematically followed, the M-CHAT/F 
was less accurate in detecting ASD than in previous studies. Disparities in screening rates and 
accuracy were evident in traditionally underrepresented groups. Future research should focus 
on the development of new methods that detect a greater proportion of children with ASD and 
reduce disparities in the screening process. 
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Although autism spectrum disorder 
(ASD) manifests in the first few years 
of life, the average age of diagnosis 
remains older than 4 years of age’ 
and is even later for children of color 
and those from rural and lower- 
income backgrounds.*” Improving 
early diagnosis is critical because it 
affords children access to earlier 
intervention, which has been shown 
to significantly improve outcomes.*° 
The American Academy of Pediatrics 
(AAP)’ recommends universal 
screening for ASD at 18 and 

24 months to facilitate earlier 
identification. However, the US 
Preventive Services Task Force 
concluded that there is insufficient 
evidence to recommend universal 
screening, in part because of limited 
data on outcomes for children who 
screen negative and from diverse 
samples. Coupled with the lack of 
data on truly universal screening (ie, 
all children are screened rather than 
a selected subset), there are critical 
gaps in knowledge about the short- 
and long-term benefits of universal 
ASD screening.® 


The most widely used and studied 
screening tool is the Modified 
Checklist for Autism in Toddlers with 
Follow-Up (M-CHAT/F),”*° a 2-stage 
tool that includes a 23-item parent 
questionnaire and a follow-up 
interview designed to reduce false- 
positives. The Modified Checklist for 
Autism in Toddlers, Revised, with 
Follow-Up (M-CHAT-R/F), reworded 
and removed items and introduced 
new scoring criteria that recommend 
bypassing the follow-up interview for 
scores of 8+. M-CHAT/F and 
M-CHAT-R/F data are frequently 
combined because M-CHAT-R/F 
scoring criteria can be applied to 
M-CHAT/F administrations, and 
accuracy is comparable across 
versions.” Estimates of positive 
predictive value (PPV) for the 
M-CHAT/F and M-CHAT-R/F have 
varied widely (2%-65%) depending 
in part on the sample’s ASD 
prevalence.” 1113 However, PPV is 


optimized when the follow-up 
interview is used. There are few 
sensitivity, specificity, or 

negative predictive value (NPV) 
estimates because these require 
systematic follow-up of all 

children, including those who screen 
negative. 


To date, 2 large-scale studies have 
been used to conduct systematic, 
longitudinal follow-up of the Modified 
Checklist for Autism in Toddlers 
without Follow-Up (M-CHAT). 
However, both have important 
limitations that restrict 
generalizability. A study conducted in 
Malaysian maternal-child health 
clinics yielded only a 0.2% screen- 
positive rate in a sample with a 0.3% 
ASD prevalence rate.’* A large study 
of population screening in Norway 
yielded a higher screen-positive rate 
(7.4%), but the ASD prevalence was 
still low (0.3%), suggesting that the 
study’s methods failed to detect many 
children with ASD.*° Despite large 
sample sizes, neither study achieved 
universal screening (ie, 65% and 
~28%, respectively).'**° Although 
there are limitations, these first 
estimates in samples that were 
systematically followed-up suggested 
low sensitivity and PPV (34%-36% 
and 2%-47%, respectively). 


It is critical to assess screening 
accuracy within the intended 
population (eg, all children in 

a primary care population) to reduce 
bias and facilitate the generalization 
of findings to similar cohorts. 
Sensitivity and specificity are often 
presumed to be fixed and inherent to 
the measure, but in reality, these 
measures are strongly influenced by 
the sample and/or cohort in which 
they are estimated.’° Samples that 
are not ascertained through universal 
screening are likely to overrepresent 
children with parent and/or 
professional concern and/or 
underrepresent children of color and 
those from lower-income households. 
In addition, PPV and NPV are directly 
linked to the sample’s prevalence 


rate, such that PPV increases and NPV 
decreases with prevalence.’° Thus, 
without universal screening research 
and representative cohorts, we risk 
drawing incorrect conclusions about 
the accuracy of screening tools in 
real-world applications. 


Our goal with this study was to 
examine the real-world accuracy of 
universal screening for ASD by using 
an epidemiological design and long- 
term follow-up through 4 to 8 years 
of age. Screening was conducted at 
the Children’s Hospital of 
Philadelphia (CHOP), a large pediatric 
network of primary and specialty 
care services with an integrated 
electronic health record (EHR). Our 
secondary goals were to examine the 
accuracy of repeated screenings and 
the effect of child and/or family 
characteristics on screening rates and 
accuracy. 


METHODS 


Setting 


The CHOP network includes 31 
pediatric primary care sites that serve 
a diverse patient population in 
Pennsylvania and New Jersey. Of 
these, 4 sites in urban Philadelphia 
serve a racially and economically 
diverse patient population (88% 
children of color, 74% public 
insurance/Medicaid), wheras 
suburban sites are less diverse (35% 
children of color, 24% public 
insurance /Medicaid). 


The M-CHAT/F is administered 
electronically at well-child visits 
between 16 and 26 months in 
accordance with Pennsylvania’s Early 
and Periodic Screening, Diagnostic, 
and Treatment program’’ and is 
available in English and Spanish. 
Screening is automatically triggered 
at all well-child visits in this age 
range, regardless of previous 
screening results, to ensure that 
children are screened twice as 
recommended by the AAP. Given that 
not all children present for 18- and 
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24-month well-child visits, the 
M-CHAT/F can also be assessed 
manually at sick visits. Once the 
M-CHAT/F is completed, 
questionnaire results autopopulate 
into the child’s visit note along with 
a link for providers to complete the 
follow-up interview when children 
screen positive. 


CHOP provides ASD diagnostic 
services through a multidisciplinary 
program that includes developmental 
pediatrics, psychology and psychiatry, 
and neurology clinics. 


This study was approved by the CHOP 
Institutional Review Board with 
a waiver of consent. 


Study Design and Patient Population 


The cohort included all children who 
presented for a well-child visit (1) 
between 16 and 26 months of age, (2) 
at a CHOP site where universal 
screening had been initiated, and (3) 
between January 2011 and July 2015 
to allow for longitudinal follow-up 
through =4 years. If the child’s first 
screening occurred before initiation 
of universal screening and the second 
occurred after universal screening 
initiation, the first screening was 
included to accurately represent 
first and second screens. Screenings 
at sick visits were also included 
(0.3% of M-CHAT/F administrations). 
When multiple M-CHAT/Fs 

were completed, the first 
administration was used unless 
otherwise noted. 


Preliminary analyses of screening 
rates and results included the entire 
cohort. Children without a primary 
care visit at =4 years of age were 
excluded from diagnostic outcome 
analyses because of insufficient 
length of follow-up. Four years was 
chosen given recent estimates of the 
median age of diagnosis for ASD."® 
Those without a documented 
language of English or Spanish 

(n = 82; 0.004%) were also excluded 
because the M-CHAT/F was not 
available in other languages. 


The subgroup whose visits 

closely followed AAP guidelines 
was also examined; these children 
were screened during primary care 
visits at 18 months (+2) and 

24 months (+2) with =3 months 
between screenings. 


Measures and EHR Methods 
Demographic Data 


Demographics, gestational age, and 
insurance payer for the screening 
visit were extracted from the EHR. 
Federal information processing 
standard codes were linked to census 
tract-level data to generate estimates 
of median income, and a median split 
was performed. Language was coded 
as English only (ie, only English was 
documented) or other language (ie, 
documentation of any non-English 
language). 


Diagnostic Data 


Diagnoses were extracted from visit 
diagnoses and problem lists (a 
comprehensive list of active and 
relevant past diagnoses). All available 
data were used to determine 
diagnostic outcome. As a result, the 
length of follow-up period varied 
across children (although all children 
included in these analyses had follow- 
up through at least 4 years). 


Children were considered to have 
ASD if an ASD diagnosis appeared in 
the EHR more than once or was 
provided by a specialist because these 
criteria have been associated with the 
greatest accuracy in other large 
health care systems.’”*° For example, 
when comparing EHR diagnostic 
codes and manual chart review, 
Coleman et al?’ found that =2 ASD 
diagnoses yielded a PPV of 87%; 
specialist diagnoses were also 
associated with higher odds of 

a confirmed diagnosis. 


Children were considered to have 

a non-ASD disorder and/or delay if 
they did not meet ASD classification 
criteria described above but had =1 
code from 1 of the following 
categories: attention-deficit/ 


hyperactivity disorder (ADHD) and 
related behaviors, anxiety disorder 
and related behaviors, disruptive 
behavior disorder and related 
behaviors, developmental delay, 
language disorder and/or delay, 
motor disorder and/or delay, sensory 
processing difficulty, and social delay 
without ASD. See Supplemental 
Table 5 for specific diagnostic 

codes used. 


Statistical Analyses 


Screening rates and results were 
summarized with percentages; x’ 
analyses and odds ratios (ORs) were 
calculated for subgroup comparisons. 
Sensitivity, specificity, PPV, and NPV 
were calculated from 2 X 2 
contingency tables. Effect sizes for the 
proportion comparisons (Cohen's h) 
and measures of statistical 
significance (2-sample tests of 
proportion) were used to estimate 
differences in M-CHAT/F accuracy 
across subgroups. Emphasis was 
placed on interpreting effect sizes 
rather than P values alone given the 
likelihood of statistical significance 
for trivial differences in this large 
cohort. Thus, only results with 
Statistically significant comparisons 
and effect sizes =0.20 are reported as 
meaningful. Finally, Kaplan-Meier 
survival curves estimated the 
cumulative probability of ASD 
diagnosis across time since screening. 
The log-rank test was used to 
compare survival curves between 
children who screened negative and 
positive to detect differences in mean 
time to diagnosis by M-CHAT/F 
outcome. 


RESULTS 


Screening Rates 


A total of 25999 children had 42 973 
eligible visits during the study period 
(see Table 1 and Fig 1). A total of 
23 634 children (90.9%) were 
screened and 50.4% were screened 
more than once. White children were 
screened more often than other racial 
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TABLE 1 Cohort Demographics 
Entire Cohort 


Sex, n (%) 

Female 

Male 
Gestational age, n (%) 

Full term, 37+ wk 

Premature, <57 wk 

Gestational age not documented in the EHR 
Language documented in the EHR, n (%) 

English only 

Language other than English 

No language documented in the EHR 
Race, n (%) 

White 

Black 

Asian 

Other or multiple races 

Race not documented in the EHR 
Ethnicity, n (%) 

Not Hispanic or Latino 

Hispanic or Latino 

Ethnicity not documented in the EHR 
Insurance payer at screening visit 

Private insurance 

Public insurance/Medicaid 

Self-pay or other 
Type of primary care site at first visit, n (%) 

Urban, n = 4 sites 

Suburban, n = 27 sites 
Median income, $, (SD)? 


N = 29999 


12905 (48.3) 
13 446 (51.7) 


20 249 (77.9) 
2870 (11.0) 
2880 (11.1) 


10958 (42.1) 
15041 (57.9) 
09 096.06 (52 579.09) 


a Missing for 17 children because the median income was unavailable for the census tract. 


groups (see Table 2). Children with 
English-only exposure, higher 
incomes, private insurance, and from 
suburban primary care sites were 
also screened more often, and 


Entire cohort: 
CHOP primary care patients 
seen at 16 to 26 months of age 
N = 25999 


premature children were screened 
less often. Only 47.8% were screened 
at 18 and 24 months largely because 
of failure to attend both well-child 
visits. Children who received 2 


Screened cohort: 
Completed autism screening 


Did not complete autism 
screening 
n = 2365 (9%) 


No diagnostic outcome data 
available 
n= 3197 (14%) 


Excluded from accuracy analyses; 


(M-CHAT/F) 
n = 23634 (91%) 


Diagnostic outcome data 
available (visit at 4+ years) 
n = 20437 (86%) 


Final screened cohort with 


M-CHAT/F not available 


in family language 
n = 82 (.004%) 


outcome data 
n = 20375 (99.7%) 


FIGURE 1 
Cohort flowchart. 


screenings according to the AAP 
schedule were more likely to be white 
and non-Hispanic, from a suburban 
site, and have English-only exposure, 
higher incomes, and private insurance 
(see Table 2). 


Screen-Positive Rates 


When considering the first M-CHAT/ 
F, 9.5% (n = 2256) screened positive 
on the 23-item questionnaire, a rate 
comparable to other large-scale US- 
based screening studies.”** Of those 
that screened positive, 88.7% 

(n = 2002) required the follow-up 
interview (ie, scores of 3-7) and 
41.2% (n = 825) were administered 
it. Almost all (n = 782; 94.8%) no 
longer screened positive after the 
follow-up interview. Of note, these 
numbers reflect all screened children 
(including those without follow-up 
data), so they differ somewhat from 
Fig 2, which only includes children 
included in accuracy analyses (ie, 
screened with follow-up data). 


For accuracy analyses, children who 
screened negative after the 
questionnaire or follow-up interview 
were considered screen negatives. 
Those who continued to screen 
positive after the interview were 
considered screen positives. Children 
who screened positive on the 
questionnaire but did not receive the 
follow-up interview were also 
considered screen positives. 
Excluding this group would introduce 
substantial bias to the cohort because 
there were demographic and clinical 
differences between children who did 
and did not receive the follow-up 
interview (see below). Furthermore, 
the positive questionnaire results 
were available to providers to base 
clinical action on (even in the absence 
of the follow-up interview), and many 
were referred after an incomplete 
M-CHAT/F screening (K.W., W.G., A.B., 
et al, unpublished data). 


This approach resulted in a final 
screen-positive rate of 6.2%. Children 
of color, those from lower-income 
households, with public insurance/ 
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TABLE 2 Screening and Follow-up Rates by Child and Family Characteristics 


Entire cohort (N = 25999) 

By sex 
Female (n = 12553) 

Male (n = 13 446) 

By gestational age 
Full term, 37+ wk (n = 20 249) 
Premature, <37 wk (n = 2870) 

By language documented in the EHR 
English only (n = 24371) 

Language other than English (n = 1271) 

By race 
White (n = 11118) 

Black (n = 9497) 
Asian (n = 1109) 
Other or multiple races (n = 4232) 

By ethnicity 
Not Hispanic or Latino (n = 23 983) 
Hispanic or Latino (n = 1929) 

By insurance payer at screening visit 
Private insurance (n = 14087) 
Public insurance/Medicaid (n = 11 769) 

By primary care site type 
Suburban (n = 10041) 

Urban (n = 10958) 

By median income 
Higher income (n = 12991) 

Lower income (n = 12991) 


Screened at Least Once 


No. (%) 
235 628 (90.9) 


11375 (90.6) 
12255 (91.1) 


18 567 (91.7) 
2429 (84.6) 


22221 (91.2) 


OR (95% CI) 


0.94 (0.86-1.02) 


2.00 (1.79-2.24)? 


2.02 (1.73-2.36)? 


Screened at 18 and 24 mo 


No. (%) 
12418 (47.8) 


0956 (47.5) 
6482 (48.2) 


10051 (49.6) 
1057 (36.8) 


11800 (48.4) 


1063 (83.6) — 454 (35.7) 
10761 (96.8) — 7151 (64.3) 
7924 (83.4) 5.98 (5.32-6.74)? 2637 (27.8) 
1016 (91.6) 2.76 (2.18-3.50)? 566 (51.0) 
3884 (91.8) 2.70 (2.32—3.14)° 2039 (48.2) 


21811 (90.9) 
1745 (90.4) 


15 437 (95.4) 
10086 (85.7) 


14667 (97.5) 
8961 (81.8) 


12571 (96.8) 
11049 (85.1) 


1.07 (0.92-1.25) 


3.45 (3.79-3.13)? 


8.77 (7.81-9.80) ° 


5.26 (4.70—5.86)° 


11544 (48.1) 
856 (45.5) 


8668 (61.5) 
3743 (51.8) 


2917 (26.6) 
9501 (65.2) 


4249 (52.7) 
8167 (62.9) 


OR (95% CI) 


0.96 (0.92-1.01) 


1.69 (1.56-1.83)? 


1.69 (1.50-1.90)? 


4.69 (4.42—4.98)° 
1.73 (1.53-1.96)° 
1.94 (1.81—2.08)° 


1.21 (1.10-1.33)? 


3.42 (3.26-3.61)? 


4.72 (4.48-5.00)? 


3.48 (3.31-3.67)° 


CHOP Patient Through at Least 4 y 
of Age (ie, Follow-Up Data 
Available) 


No. (%) 
22 392 (86.1) 


10 841 (86.4) 
11551 (89.9) 


17 608 (87.2) 
2454 (89.9) 


21244 (87.2) 
1071 (84.3) 


9710 (87.3) 
8236 (86.7) 
888 (80.1) 
5027 (83.3) 


20 724 (86.4) 
1655 (84.7) 


12212 (87.1) 
9955 (85.0) 


15 101 (87.1) 
9291 (84.8) 


11 306 (87.0) 
11074 (85.2) 


OR (95% Cl) 


1.04 (0.97-1.11) 


1.15 (1.03-1.29) 


1.27 (1.09-1.48)? 


1.05 (0.97-1.15)? 
1.72 (1.47-2.01)8 
1.38 (1.25-1.52)° 


1.15 (1.01-1.31)? 


1.19 (1.11-1.28)? 


1.21 (1.13-1.30)? 


1.16 (1.08-1.25)° 


The last group listed is the reference group for ORs, except for comparisons by race. Here, black, Asian, and other or multiple races are the reference groups, respectively, and compared 
with the white group. Some subgroups do not add to 25 999 participants because of missing data. Cl, confidence interval; —, not applicable. 
a Indicates screening percentages that are different by subgroup at P < .05 after Bonferroni correction. 


Medicaid and non-English exposure, 
and those seen in urban practices 
screened positive more frequently, as 
did boys and premature children (see 
Table 2). 


Screen-Positive Rates by Age 


Older toddlers (21-26 months) 
screened positive more often (8.9%) 
than younger toddlers (16-20 
months; 5.5%) on the first screening 
(see Table 3). However, screen- 
positive rates were somewhat higher 
at 18 months (4.2%) than at 

24 months (3.5%) in the subgroup 
screened twice; 6.4% screened 
positive on 1 or both screenings. 


Outcome Diagnostic Data 


Most screened children (n = 20 437; 
86.5%) continued to receive CHOP 
primary care at =4 years and were 
included in M-CHAT/F accuracy 


analyses because outcome diagnostic 
data were available. ASD prevalence 
was 2.2%, which is comparable to 
recent prevalence estimates in nearby 
New Jersey. ’“* A total 62.8% received 
an ASD diagnosis by a specialist (of 
these, 94.7% also had an ASD 
diagnosis documented by a primary 
care provider); the remaining 37.2% 
only had a diagnosis made or 
documented by a primary care 
provider. 


Overall, 36.4% had other delays and/ 
or concerns, which included codes 
related to development (10.6%), 
language (23.0%), behavior (8.4%), 
motor (7.9%), ADHD and related 
behaviors (2.9%), anxiety and related 
behaviors (1.8%), sensory processing 
(0.4%), and social delays without ASD 
(0.6%; categories are not mutually 
exclusive). 


Accuracy Analyses 


The M-CHAT/F’s sensitivity to detect 
ASD was 38.8%, and its specificity 
was 94.9%. PPV was 14.6% and NPV 
was 98.6% (see Table 4, Fig 2, and 
Fig 3). The M-CHAT/F’s accuracy in 
detecting any documented delay and/ 
or concern (including ASD) was as 
follows: sensitivity was 11.8%, 
specificity was 97.4%, PPV was 
72.4%, and NPV was 65.9%. 


Effect of Age and Multiple Screenings 


Across all first-time screenings, 
screenings at older ages (21-26 
months) were more sensitive than at 
younger ages (16-20 months; 48.8% 
vs 35.1%). In the AAP subgroup of 
children screened twice, the second 
screening at 24 months was 
marginally more sensitive (39.8% vs 
31.8%) and had higher PPV (24.7% 
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Screened cohort with 
outcome data 
N = 20375 


Screened negative on the 
M-CHAT questionnaire 
n = 18512 (90.9%) 


M-CHAT score = 8+ 
(bypass interview) 


n = 201 (10.8%) 


Screen negative Screen positive Screen positive Screen positive 
n = 18512 n = 201 n = 967 n = 34 


FIGURE 2 


Screened positive on the 
M-CHAT questionnaire 
n = 1863 (9.1%) 


M-CHAT score = 3 to 7 
(follow-up interview indicated) 


n = 1662 (89.2%) 


Follow-up interview 
not administered 
n = 967 (58.2%) 


Continued to screen positive 
after follow-up interview 


n = 34 (4.9%) 


M-CHAT/F results for screened cohort with outcome data. 


vs 16.4%) than the first screening at 
18 months. Combining results from 
18- and 24-month screenings yielded 
greater sensitivity than either 
screening alone (51.1%). See Table 4. 


Effect of Child and/or Family 
Characteristics 


Despite comparable sensitivities 
across racial groups, specificity and 
PPV were higher in white children 
(97.9%; 24.0%) compared with black 
children (91.7%; 11.7%), Asian 
children (90.4%; 10.8%), and those 
from other or multiple racial groups 
(93.8%; 13.4%). Differences were not 
observed between black, Asian, and 
other or multiple racial groups, or by 
ethnicity. 


Higher specificity and PPV were 
observed in children with English- 
only exposure compared with 


children with non-English exposure 
(95.2% vs 86.9%; 15.3% vs 8.5%) as 
well as children from higher- versus 
lower-income families (97.0% vs 
92.3%; 20.4% vs 11.8%). The same 
pattern was observed for insurance 
payer because children with private 
insurance had higher specificity 
(97.6% vs 91.0%) and PPV (22.1% 
vs 11.3%). This pattern was 

also observed for practice type 
because specificity was higher 

in children screened in suburban 
sites (96.8% vs 91.5%) as was 

PPV, although this effect size fell 
below the cutoff (h = 0.18; 18.5% vs 
12.0%). 


PPV was higher in boys than in girls 
(19.9% vs 7.7%). Sensitivity was 
higher in children born premature 
(54.3% vs 35.8%), but specificity was 
lower (89.3% vs 95.4%). 


Follow-up interview 
administered 
n = 695 (41.8%) 


Screened negative after 


follow-up interview 
n = 661 (95.1%) 


Screen negative 
n = 661 


Effect of Follow-up Interview 
Administration 


Children requiring the follow-up 
interview (ie, scores of 3-7) who 
received it were more likely to be full 
term (P = .007; OR = 1.41), have 
lower incomes (P < .001; OR = 1.65), 
and be from urban practices (P < 
.001; OR = 1.60) than those who did 
not receive the follow-up interview. 
Black children were also more likely 
than white (P = .001; OR = 1.48), 
Asian (P = .06; OR = 1.40), or children 
of other or multiple races (P < .001; 
OR = 1.64) to receive the follow-up 
interview. Children without ASD were 
also more likely to receive the follow- 
up interview than children with ASD 
(P = .02; OR = 1.54). 


PPV was examined separately on the 
basis of the presence or absence of 
follow-up interview results. PPV was 
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Screened cohort with an 
outcome diagnosis of ASD 
n= 454 


Screened negative on the 
M-CHAT questionnaire 
n = 249 (54.8%) 


Screened positive on the 
M-CHAT questionnaire 
n = 205 (45.2%) 


M-CHAT score = 8+ 
(bypass interview) 
n= 70 (34.1%) 


M-CHAT score = 3 to 7 
(follow-up interview indicated) 
n = 135 (65.9%) 


Screen negative 
n= 249 
(54.8% of ASD) 


Screen positive 
n= 70 
(15.4% of ASD) 


FIGURE 3 


Follow-up interview 
not administered 
n = 93 (68.9%) 


Continued to screen positive 
after follow-up interview 


n = 13 (31.0%) 


Screen positive 
n= 93 
(20.5% of ASD) 


n=13 


M-CHAT/F results for screened cohort with outcome diagnosis of ASD. 


34.8% in children with 

a questionnaire score of 8+ (ie, 
follow-up interview bypassed, 

n = 201). PPV was 38.2% in children 
who continued to screen positive 
after receiving the follow-up 
interview (n = 34). PPV was 9.6% 

in those that did not receive the 
interview (n = 967). Other metrics 
could not be calculated separately by 
follow-up results because they are 
calculated by using screen negatives, 
and it is not possible to know how 
many of the 967 children who did not 
receive the interview would have 
screened negative. 


Excluding all follow-up interview data 
(ie, M-CHAT/F questionnaire results), 
sensitivity was 45.2%, specificity was 
91.7%, PPV was 11.0%, and NPV 
was 98.7%. 


M-CHAT/F Results and Time to 
Diagnosis 


Kaplan-Meier survival curves 
revealed that the mean time to 
diagnosis was significantly shorter for 
children with ASD who screened 
positive than for those who screened 
negative (mean difference = 7.45 
months; P < .001). 


DISCUSSION 


In this study we examined the 
M-CHAT/F’s accuracy within 

a universally screened cohort. The 
results revealed high screening rates, 
achieved through robust EHR support 
for screening. Systematic follow-up of 
children who screened positive and 
negative allowed an estimation of 
sensitivity, specificity, PPV, and NPV. 
The M-CHAT/F’s sensitivity to detect 


Screen positive 


(2.9% of ASD) 


Follow-up interview 
administered 
n = 42 (31.1%) 


Screened negative after 
follow-up interview 
n = 29 (69.0%) 


Screen negative 
n= 29 
(6.4% of ASD) 


ASD was just 39%, indicating that 
the majority of children later 
diagnosed with ASD screened 
negative. PPV was just 15%, an 
estimate consistent with recent 
large (but not universally screened) 
studies conducted in Norway and 
Malaysia.’*'° However, this 
estimate was much lower than that 
found in some US-based studies 
conducted in research settings, likely 
partially because of different 
prevalence rates across studies.”** 
PPV improved substantially when 
any diagnosis or concern was 
considered (72%). Specificity and 
NPV for ASD were high (95% and 
99%, respectively), but it is important 
to remember that with low 
prevalence and screen-positive 
rates, specificity and NPV will tend 
toward high rates. 
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TABLE 3 M-CHAT/F Screen-Positive Rates by Child and Family Characteristics 


Screened cohort (n = 23628) 
By age at first screening 
16—20 mo (n = 18585) 
21—26 mo (n = 5043) 
By multiple screenings 
First screening at ~18 mo (n = 11624) 
Second screening at ~24 mo (n = 11624) 
By sex 
Female (n = 11375) 
Male (n = 12253) 
By gestational age 
Full term, 37+ wk (n = 18567) 
Premature, <37 wk (n = 2429) 
By language documented in the EHR 
English only (n = 22 221) 
Language other than English (n = 1063) 
By race 
White (n = 10 761) 
Black (n = 7924) 
Asian (n = 1016) 
Other or multiple races (n = 3884) 
By ethnicity 
Not Hispanic or Latino (n = 21 811) 
Hispanic or Latino (n = 1743) 
By insurance payer at screening visit 
Private insurance (n = 13 437) 
Public insurance/Medicaid (n = 10 086) 
By primary care site type 
Suburban (n = 14667) 
Urban (n = 8961) 
By median income 
Higher income (n = 125/71) 
Lower income (n = 11049) 


Screened Positive on M-CHAT/F, 


No. (%) 
1474 (6.2) 


1027 (9.5) 
447 (8.9) 


018 (4.2) 
440 (3.5) 


635 (5.6) 
859 (6.8) 


1022 (5.5) 
502 (12.4) 


1301 (5.9) 
159 (15.0) 


319 (3.0) 
752 (9.5) 
108 (10.6) 
295 (7.6)° 


1305 (6.0) 
167 (9.6) 


427 (3.2) 
1038 (10.3) 


610 (4.2) 
864 (9.6) 


496 (3.9) 
977 (8.8) 


OR (95% Cl) 


1.66 (1.48-1.87)° 


0.84 (0.74-0.96)? 


1.24 (1.12—1.38)° 


2.44 (2.13-2.80)? 


9.83 (2.37-3.38)? 


3.45 (3.01-3.94)? 
3.90 (3.11-4.90)? 
2.70 (2.29-3.16)? 


1.67 (1.41-1.97)° 


3.50 (3.11-3.93)? 


2.46 (2.21-2.74)° 


9.36 (2.11-2.64)? 


The first group listed is the reference group for ORs. Some subgroups do not add to 23 628 participants because of missing data. A comparison of the first and second screening was 
conducted in subsample with 2 screenings according to AAP guidelines (n = 11624 children). Cl, confidence interval; —, not applicable. 
a Indicates screen-positive rates that are different by subgroup at P < .05 after Bonferroni correction. 


Although the M-CHAT/F identified 
fewer children with ASD than 
expected, those who did screen 
positive received an ASD diagnosis 

7 months earlier than those who 
screened negative. This suggests that 
a positive M-CHAT/F screening may 
have contributed to an earlier 
diagnosis for children with ASD in 
this cohort. However, continued 
research is needed to understand the 
specific effect of a positive screen on 
age of diagnosis. 


The M-CHAT/F was significantly 
more sensitive at older ages (49% at 
21-26 months) than at younger ages 
(35% at 16-20 months). However, 
PPV did not improve with age (16% 
vs 14%), which is in contrast to 


findings from some previous 
studies.'* PPV did improve for 
repeated screenings; 25% of children 
who screened positive at the second 
screening had ASD (regardless of 
results of the first screening) 
compared with 17% for the first 
screen. These results highlight the 
potential importance of screening 
twice as well as the difficulty of 
accurate screening at ~18 months. 


Sensitivity was higher, but specificity 
was lower for premature children 

compared with those born full term, 
consistent with several other studies 
in very premature children.**- 7° PPV 
was lower in girls (8%) than in boys 
(20%), but it is unclear whether this 
was because of delayed diagnosis for 


some girls or poorer M-CHAT/F 
performance in girls, but future 
research should be used to examine 
sex differences in each stage of 
detection, from screening to 
diagnosis. 


Although electronic screening yielded 
high screening rates, when children 
were missed, they were more likely to 
be children of color, from lower- 
income households, seen in an 
urban practice, receive public 
insurance/Medicaid, and be exposed 
to a language other than English. 
These same children were also less 
likely to present for 2 well-child 
visits, resulting in disparities in 
receiving care according to AAP 
guidelines. 
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TABLE 4 M-CHAT/F Accuracy in Detecting ASD by Child and Family Characteristics 


Screened cohort with outcome data (n = 20 375) 
By age at first screening 
16—20 mo 
21—26 mo 
Comparison of accuracy by age 
By multiple screenings 
First screening only at ~18 mo without regard for results of second 
screening 
Second screening only at ~24 mo without regard for results of second 
screening 
Comparison of accuracy by screening, first vs second 
First or second screening positive 
Comparison of accuracy by no. of screenings, first screening vs both 
screenings 
By sex 
Female 
Male 
Comparison of accuracy by sex 
By gestational age 
Full term, 37+ wk 
Premature, <57 wk 
Comparison of accuracy by gestational age 
By language documented in the EHR 
English only 
Language other than English 


Sensitivity (95% 
Cl) 


58.8 (54.5-43.3) 


35.1 (30.0-40.3)° 
48.8 (39.6-57.7)° 


P = 009, h = 0.28? P< 001, h = 0.11 


31.8 (25.9—37.7)° 
39.8 (33.5—46.0) 


P= 07, h = 0.17 
51.1 (44.7-57.4)? 


P< 001, h = 0.59? P< .001, h = 0.09 


59.6 (30.1-49.1) 
58.0 (535.5-43.6) 
P= 80, h = 0.02 


35.8 (30.8-40.8)° 
54.3 (42.6-66.0)° 


P = 004, h = 0.37° 


58.0 (535.9-43.1) 
43.9 (23.2-63.7) 


Specificity (95% 
Cl) 


94.9 (94.5-95.2) 
95.4 (99.1-95.7) 
92.3 (92.0-93.5) 
96.6 (96.3-97.0) 
97.4 (97.1—97.7) 


P = 001, h = 0.04 
94.8 (94.4—95.2) 


95.1 (94.7—95.5) 
94.6 (94.2—95.1) 
P = 19, h = 0.02 


95.4 (95.1—95.8)° 
89.3 (87.9—90.6)° 


P < 001, h = 0.24° 


95.2 (94.9—95.5)° 
86.9 (84.6-89.2)° 


P < .001, h = 0.30° 


97.9 (97.6—98.1)° 
91.7 (91.0-92.3)° 


PPV (95% Cl) 


14.6 (12.6-16.6) 


15.9 (11.6-16.2) 
16.4 (12.6-20.3) 
P = 25, h = 0.07 


17.0 (15.5—20.4) 
24.7 (20.4—29.1) 


P = .006, h = 0.19 
17.6 (14.8-20.5) 
P = .11, h = 0.02 


7.7 (5.4—10.0)° 
19.9 (16.9-22.9)° 
P< 001, h=0.36° 


14.8 (12.4-17.2) 
10.0 (10.6-19.4) 
P = .90, h = 0.01 


15.3 (13.2—17.5)° 
8.5 (3.4-13.5)? 
P= 046 h = 0.21° 


24.0 (18.9-29.2)° 
11.7 (9.2-14.2)° 


NPV (95% CI) 


98.6 (98.4—98.7) 
98.6 (98.4—98.8) 
98.4 (98.0-98.8) 
P = .41, h = 0.01 
98.5 (98.3-98.7) 


98.7 (98.5-98.9) 


P=20,0= 0.02 
98.9 (98.7—-99.1) 
P= 01, h = 0.04 


99.4 (99.2-99.5) 
97.8 (97.5—98.1) 
P< 001, h = 0.14 


98.6 (98.4—98.7) 
98.2 (97.6—98.8) 
P = .ő1, h = 0.02 


98.6 (98.4—98.7) 
98.2 (97.3-99.2) 
P = 46, h = 0.05 


98.9 (98.6—-99.1) 
98.3 (98.0-98.6) 


Comparison of accuracy by language P = 65, h = 0.10 
By race 
White 37.7 (30.4—45.1) 
Black 40.7 (33.5—47.8) 
Comparison of accuracy by race vs white P= 08, h = 0.06 
Asian 30.0 (13.6—46.4) 
Comparison of accuracy by race vs white P= 42,h=0.16 
Other or multiple races 40.0 (28.9-51.1) 
Comparison of accuracy by race vs white P = 73, h = 0.05 


By ethnicity 

Hispanic or Latino 

Not Hispanic or Latino 

Comparison of accuracy by ethnicity 
By insurance payer at screening visit 

Public insurance/Medicaid 

Private insurance 

Comparison of accuracy by insurance 
By primary care site type 

Urban 

Suburban 

Comparison of accuracy by site type 
By median income 

Lower income 

Higher income 

Comparison of accuracy by income 


Cl, confidence interval. 


42.1 (26.4-97.8) 
58.0 (33.8-43.1) 
P = .66, h = 0.07 


42.8 (56.5—49.5) 
54.4 (28.2—40.5) 
P= 07, h = 0.17 


40.2 (55.6—46.8) 
37.0 (31.4-43.6) 
P = .06, h = 0.06 


59.0 (52.8-40.2) 
58.7 (52.1—45.2) 
P= 94 h = 0.01 


a These comparisons are statistically significant (P < .05) and clinically meaningful (h = 0.20). 


Despite these disparities, screening 
rates among children from 
traditionally underrepresented 
groups were still relatively high 
(>80%). Children of color and those 
from lower-income households 
screened positive at 2 to 3 times the 


rates of white, higher-income, 
privately insured, and suburban 
children. Elevated screen-positive 
rates resulted in somewhat higher 
sensitivity to detect ASD but also 
higher false-positive rates (ie, lower 


PPV and specificity) in these groups. 


P< 001, h = 0.29° 
90.4 (88.3-92.4)° 
P< 001, h = 0.348 
93.8 (93.0-94.7)° 
P< 001, h = 0.21° 


92.2 (90.8—95.6) 
95.1 (94.7—95.4) 
P < .001, h = 0.12 


91.0 (90.4—91.6)° 
97.6 (97.3—97.9)° 
P < 001, h=0.30° 


91.5 (90.9-92.1)? 
96.8 (96.5-97.1)° 
P< 001, h = 0.23? 


92.3 (91.8-92.9)? 
97.0 (96.7-97.3)° 
P< 001, h = 0.22" 


P< 001, h =0.33° 
10.8 (4.2-17.5)° 
P = 01, h = 0.35° 
13.4 (8.9-17.9)? 
P = 003, h = 0.28° 


P = 002, h = 0.00 
97.1 (95.8-98.3) 
P < 001, h = 0.15 
98.5 (98.1-98.9) 
P = .11, h = 0.05 


12.5 (6.8—18.2) 
14.9 (12.8—17.0) 
P= 41, h = 0.07 


98.4 (97.7—99.0) 
98.6 (98.4—98.7) 
P = .56, h = 0.02 


11.3 (9.1-13.4)? 
99.1 (17.8—26.4)° 
P < 001 h = 0.29° 


98.5 (98.1-98.6) 
98.7 (98.5—98.9) 
P = .05, h = 0.05 


12.0 (9.6-14.4) 
18.5 (15.0—-21.9) 
P = 002, h = 0.18 


98.1 (97.8-98.5) 
98.8 (98.6—-99.0) 
P< 001, h = 0.05 


11.8 (9.5-14.0)° 
20.4 (16.5—24.4)° 
P< 001, h = 0.24? 


98.5 (98.0-98.6) 
98.8 (98.6—-99.0) 
P = .006, h = 0.04 


It is suggested by these data that 
disparities in age of diagnosis are 
likely preceded by disparities in 
screening rates and differential 
accuracy of screening tools for 
children from underrepresented and 
underresourced groups. Future 
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research should attempt to 
disentangle the effects of race and/or 
ethnicity, income, language, and 
primary care setting to better 
understand the role of child-, family-, 
and practice-level variables on 
screening. 


With regard to the follow-up 
interview, pediatricians did not 
complete the interview systematically 
but instead may have used clinical 
judgement when deciding when to 
administer. Children who received the 
follow-up interview were significantly 
less likely to have ASD, and when 
children received the follow-up, 
almost all (95%) screened negative. 
Thus, pediatricians may have used 
the interview to confirm a clinical 
opinion of “not ASD” and chosen to 
skip the interview and go straight to 
referral when they suspected ASD. 
Additional research is needed to 
disentangle differences in follow-up 
interview rates by race and income; 
this may have been a clinical 
adaptation to artificially high screen- 
positive rates or it may incorrectly 
delay diagnosis for these children. 
However, the follow-up interview did 
appear to reduce false-negatives (ie, 
improve PPV), underscoring the 
importance of this step of the 
screening process. 


There are limitations to the current 
study, largely surrounding the real- 
world nature of this cohort. Although 
this epidemiological study 
represented all children within the 
CHOP primary care network, findings 
may not generalize to other 
populations, particularly those with 
less access to ASD specialty care. 
Diagnoses were given in real-world 
clinical settings rather than through 


rigorous research studies. Diagnostic 
information was also only available 
through 4 to 8 years of age, so the 
M-CHAT/F’s accuracy may differ as 
children in this cohort age. 


The M-CHAT/F, rather than the 
M-CHAT-R/F, was used, although 
accuracy of these 2 versions is 
comparable.” As indicated above, not 
all eligible children received the 
follow-up interview, which likely 
downwardly biased specificity and 
PPV and upwardly biased sensitivity 
and NPV in that some would 

have screened negative if they 

had received the follow-up 
interview. Thus, this study cannot 
estimate the accuracy of perfect 
M-CHAT/F administration but 
instead provides critical information 
on how well this tool detects ASD in 
a real-world, universally screened 
cohort. 


CONCLUSIONS 


These results suggest that universal 
screening in primary care is possible 
when supported by electronic 
administration and EHR integration. 
However, universal screening and 
systematic follow-up revealed low 
accuracy of the M-CHAT/F, 
particularly for children of color and 
those from lower-income households. 
Although some may interpret these 
findings as evidence against universal 
screening, we caution against this 
interpretation given the earlier age of 
diagnosis for screen positives. 
Instead, results suggest that 
augmentative screening methods 
should be developed to detect more 
children through universal screening 
efforts and reduce disparities. 


Promising new methods include 
parent-report tools that are 
supported by picture or video 
models”° and direct data 

gathering methods that leverage 
technological advances in computing 
and machine learning.*”*° However, 
any new method should be tested in 
cohorts that are universally screened 
and systematically followed-up to 
reduce the bias associated with 
screening and following selected 
populations. 
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Trends in Psychiatric Emergency 
Department Visits Among Youth and 
Young Adults in the US 


Luther G. Kalb, PhD,?® Emma K. Stapp, PhD,° Elizabeth D. Ballard, PhD,° Calliope Holingue, MPH,? Amy Keefer, PhD,>* 
Anne Riley, PhD‘ 


BACKGROUND: Visits to the emergency department (ED) for psychiatric purposes are an indicator 
of chronic and acute unmet mental health needs. In the current study, we examined if 
psychiatric ED visits among individuals 6 to 24 years of age are increasing nationwide. 


METHODS: ED data came from the 2011-2015 National Hospital Ambulatory Medical Care 
Survey, a national survey of ED visits across the United States. Psychiatric ED visits were 
identified by using the /nternational Classification of Diseases, Ninth Revision and reason-for- 
visit codes. Survey-weighted logistic regression analyses were employed to examine trends in 
as well as correlates of psychiatric ED visits. Data from the US Census Bureau were used to 
examine population rates. 


RESULTS: Between 2011 and 2015, there was a 28% overall increase (from 31.3 to 40.2) in 
psychiatric ED visits per 1000 youth in the United States. The largest increases in psychiatric 
ED visits per 1000 US youth were observed among adolescents (54%) and African American 
(53%) and Hispanic patients (91%). A large increase in suicide-related visits (by 2.5-fold) was 
observed among adolescents (4.6-11.7 visits per 1000 US youth). Although psychiatric ED 
visits were long (51% were =3 hours in length), few (16%) patients were seen by a mental 
health professional during their visit. 


CONCLUSIONS: Visits to the ED for psychiatric purposes among youth are rising across the United 
States. Psychiatric expertise and effective mental health treatment options, particular those 
used to address the rising suicide epidemic among adolescents, are needed in the ED. 
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ARTICLE 


TABLE 1 Age-Related Characteristics of Psychiatric ED Visits Among Youth in the United States 


Count of total visits (weighted) 

Count of psychiatric ED visits (weighted) 

Estimated youth in the United States, 
2011—2015 


Psychiatric ED visits, % of all visits (count) 


2011 
2012 
2015 
2014 
2019 


Psychiatric ED visits per 1000 US youth 
2011 
2012 
2013 
2014 
2015 
Region, % of psychiatric ED visits 
Northeast 
Midwest 
South 
West 
Sex, % 
Female 
Male 
Race, % 
Non-Hispanic White 
Non-Hispanic African American 
Hispanic 
Non-Hispanic other 
Insurance, % 
Private 
Public 
Self-pay 
Other 
Visit on a weekend, % 
No 
Yes 
Arrival time, % 
Morning (<11 Am) 
Afternoon (11 am—4 pm) 
Evening (5 PM—12 am) 
Urgent, % 
No 
Yes 
Arrive by ambulance, % 
No 
Yes 
Length of visit, % 
<90 min 
90-179 min 
180—559 min 
=6 h 
Mental health provider seen, % 
No 
Yes 
Admit to hospital, % 


6-11 y 


50 497 171 


820 616 


123 158 640 


2.0 
(155 225) 
2.0 
(152 255) 
2.8 
(179 852) 
2.5 
(181 471) 
2.0 
(198 855) 


0.4 
0.4 
7.3 
7.3 
8.0 


29.6 
22.4 
29.5 
18.7 


28.4 
71.6 


00. 
22.9 
16.5 
49 


52.2 
06.4 
6.7 
4.6 


85.0 
16.5 


18.6 
28.5 
09.0 


54.5 
69.7 


89.9 
14.1 


18.5 
20,1 
56.9 
18.8 


84.5 
10.4 


19-17 y 


59 595 222 
ó 881 608 
129 212 948 


9.0 (727 111) 


8.2 (607 112) 


9.4 (675 533) 


9.1 (799 863) 


15.2 
(1 116 189) 


28.9 
24.2 
27.0 
30.2 
44.6 


20.6 
29.9 
55.9 
19.9 


06./ 
43.3 


08.6 
19.4 
19.8 
2.1 


58.6 
92./ 
0.9 
2.8 


76.7 
25.5 


20.1 
O1./ 
48.2 


21.9 
78.1 


78.7 
21.5 


17.4 
28.5 
30.0 
25.1 


79.1 
24.9 


19-24 y 


81201715 
8 454 562 
104 601 395 


9.4 (1 659 670) 


9.7 (1579 440) 


9.8 (1907 892) 


10.7 

(1 770 647) 
12.5 

(1 924 713) 


04.0 
00.9 
48.9 
07.0 
62.5 


18.8 
24.1 
35.7 
22./ 


49.7 
00.8 


64.5 
19.7 
15.8 
2.1 


52.0 
58.0 
21.2 
8.5 


71.9 
28.1 


52./ 
29.2 
58.0 


33.0 
66.9 


11.2 
28.8 


21.9 
28.9 
28.5 
21.1 


87.4 
12.6 


Overall 


106 092 106 
15 141 586 
402 952 945 


7.8 (2 516 006) 


7.6 (2514 787) 


8.1 (2363 077) 


8.3 (2 707 981) 


10.1 
(3 239 755) 


51.3 
28./ 
29.5 
33.0 
40.2 


20.0 
20.0 
33.0 
21.0 


00.0 
00.0 


62.1 
19.8 
10.8 
2.5 


54.3 
43.6 
10.9 
6.4 


74.0 
26.0 


28.1 
29.9 
42.0 


50.1 
69.9 


20.6 
74.4 


20.4 
28.0 
29.4 
21.7 


85.0 
16.5 


It is estimated that >1 in 10 youth in 
the United States has a serious 
psychiatric disorder.’ Many of these 
children, however, never receive 
treatment.’ For those in crisis 

(eg, suicide ideation, aggression, and 
psychosis) or without accessible 
connections to the mental health 
care system, the emergency 
department (ED) has become the 
national safety net because this 
setting is obligated to treat all 
patients regardless of day or time, 
reason for referral, or availability of 
financial payment. This status as 

a frontline provider has led EDs to 
the breaking point, in which EDs 
are overcrowded and shutting their 
doors because of lack of financial 
protections.” 


Because the ED is designed as an 
urgent medical facility, this setting 
often lacks the resources required to 
identify and manage psychiatric 
populations.*” Discharge planning is 
also a challenge because the wait for 
outpatient psychiatry is long, and 
inpatient psychiatric beds are 

a dwindling resource.” ® As a result, 
ED mental health visits are associated 
with long wait times, which are due in 
part to psychiatric “boarding” (in 
which a patient waits in the ED for an 
inpatient bed).”*” 


Pediatric ED visits have steadily 
increased over the last several 
decades.'*~*® The most recent 
estimates suggest that 5% to 7% 

of all pediatric ED visits in the 
United States are related to mental 
health and/or substance abuse.*”’*® 
This is likely an underestimate 
because passive mental health 
reporting, in which providers do 
not actively screen patients for 
mental health issues, is the routine 
procedure in the ED.” The 
increased presence of pediatric 
psychiatric visits in the ED has 
become such a public health problem 
that it prompted a 2016 joint 
statement by the American Academy 
of Pediatrics and the American 
College of Emergency Physicians.”*® 
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TABLE 1 Continued 


6-11 y 
No 81.6 
Yes or transfer to another hospital 6.3 
Transfer or admit to a psychiatric 12.1 
facility 
Any psychiatric diagnosis or RFV? % 
Mood 45.4 
Behavior 49.3 
SUD" 4.5 
Psychosis 6.2 
Suicide attempt or self-injury 4.2 
Other 40.8 
Count of psychiatric diagnoses or 1.59 
RFVs, mean 
Primary diagnosis is psychiatric, % 
No 37.9 
Yes 62.1 


a Reflects the presence of any of the listed diagnoses or RFVs. 


* P< 05. 


In that report, the authors provide 


clinical recommendations regarding 


best practices when managing 
youth with mental health issues 
in the ED.”*® 


12-17 y 


14.6 
8.9 
16.4 


62.7 
19.7 
26.6 
0.1 

22.7 
ó9.7 
1.95 


25.5 
76.7 


18-24 y 


84.5 
9.2 
6.9 


48.9 
8.2 
00.7 
7.5 
8.9 
24.8 
1.39 


58.9 
61.1 


Overall 


81.5 
8.9 
9.8 


02.0 
15.7 
40.6 
6.6 
12.7 
50.2 
1.42 


54.2 
69.8 


Less is known about psychiatric ED 


use among transition-aged adults. 


Most research has been focused on 
youth (<17 years of age) or adults 
(=18 years), lumping young adults 


TABLE 2 Multivariate Analysis of Correlates of Psychiatric ED Visits Among Age Groups 


Year 
Region 
Northeast 
Midwest 
South 
West 
Sex 
Female 
Male 
Race and/or ethnicity 
White, non-Hispanic 
African American, non-Hispanic 
Hispanic 
Other 
Insurance 
Private 
Public 
Self-pay 
Other 
Visit on a weekend 
No 
Yes 
Arrival time 
Morning (<11 am) 
Afternoon (11 AM—4 PM) 
Evening (5 PMm—12 am) 
Urgent 
No 
Yes 
Length of visit 


Children 
RRR 95% Cl 
1.04 0.89 to 1.22 
Reference Reference 
0.56 0.29 to 1.09 
0.67 0.37 to 1.23 
0.49° 0.27 to 0.90 
Reference Reference 
265 1.67 to 4.19 
Reference Reference 
1.28 0.81 to 2.03 
1.27 0.64 to 2.52 
2.96 0.63 to 13.95 
Reference Reference 
1.42 0.79 to 2.57 
0.30% 0.10 to 0.92 
0.48 0.14 to 1.70 
Reference Reference 
0.57 0.15 to 0.94 
Reference Reference 
1.74 0.91 to 3.14 
235 1.34 to 4.12 
Reference Reference 
0.85 0.50 to 1.44 
0.99 0.99 to 1.00 


Adolescents 


RRR 
1.04 


Reference 
0.99 
1.04 
0.74 


Reference 
0.78 


Reference 
1.10 
1.74 
1.21 


Reference 
1.05 
0.22" 
0.26* 


Reference 
0.86 


Reference 
1.69" 
204 


Reference 
1.76 
1.00 


95% Cl 
0.94 to 1.14 


Reference 
0.67 to 1.45 
0.69 to 1.59 
0.49 to 1.12 


Reference 
0.61 to 0.99 


Reference 
0.79 to 1.59 
1.22 to 2.47 
0.54 to 2.74 


Reference 
0.74 to 1.43 
0.14 to 0.34 
0.12 to 0.54 


Reference 
0.65 to 1.18 


Reference 
1.19 to 2.39 
1.49 to 2.80 


Reference 
1.24 to 2.50 
0.99 to 1.00 


(18-24 years) with older populations 
who have substantially different 
mental health profiles.*”° 
Investigating psychiatric ED use 
among young adults is critical 
because many serious psychiatric 
disorders emerge in young adulthood, 
and both mental health and substance 
abuse-related problems are directly 
related to the top 3 leading causes of 
death for young adults (ie, injury, 
suicide, and homicide). +? 


Understanding psychiatric ED use 
among youth and young adults is 
important because the ED is an 
entryway to the mental health system 
for many children and families.*° 
These visits present a unique 
opportunity to recognize and initiate 
treatment services for previously 
unidentified mental health problems.* 
The ED is also at the front lines of the 
nation’s substance abuse and suicide 
epidemics, which uniquely situates 
them to intervene in these public 
health crises.**’2° 


For the current study, we have 3 aims. 
First, we provide updated estimates 
on changes in psychiatric ED visits in 
the United States (between 2011 and 
2015) among youth 6 to 24 years of 
age. Second, we assessed if there 
were disparate trends in psychiatric 
ED visits over time across different 
age, sex, and racial and/or ethnic 
groups. Third, we examined 
correlates of psychiatric ED visits 
across the different age, sex, and 
racial and/or ethnic groups. 
Particular attention was paid to 
suicidal-attempt and self-injury visits. 


METHODS 


Data for this study primarily came 
from the 2011-2015 National 
Hospital Ambulatory Medical Care 
Survey (NHAMCS).*° The NHAMCS is 
a cross-sectional national probability 
survey of ED visits across the United 
States. Survey sampling weights allow 
for generalization of NHAMCS 
estimates to all nonfederal, short-stay, 
and general hospital EDs across the 
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TABLE 2 Continued 


Children Adolescents 
RRR 95% Cl RRR 95% Cl 

Arrive by ambulance 

No Reference Reference Reference Reference 

Yes 0.39° 0.21 to 0.75 0.71" 0.54 to 0.93 
Mental health provider seen 

No Reference Reference Reference Reference 

Yes 1.06 0.32 to 2.11 2.08" 1.46 to 2.96 
Admit to hospital 

No Reference Reference Reference Reference 

Yes or transfer to another hospital 0.98 0.31 to 1.11 1.06 0.67 to 1.69 

Transfer or admit to a psychiatric facility 1.48 0.61 to 3.08 247 1.49 to 4.11 
Psychiatric diagnosis or RFV° 

Mood 0.83 0.50 to 1.37 1.577 1.18 to 2.08 

Behavior 7.64 4.62 to 12.62 2.53, 1.79 to 3.56 

SUDs 0.05" 0.02 to 0.13 0.41" 0.30 to 0.54 

Psychosis 0.57 0.22 to 1.47 0.67 0.44 to 1.02 

Suicide attempt or intentional self-injury 0.47 0.19 to 1.14 3.19 2.27 to 4.48 

Other 1.68 1.12 to 2.53 1.87 1.43 to 2.43 
Count of diagnoses and/or RFVs 0.86 0.61 to 1.20 1.30° 1.09 to 1.54 
Primary diagnosis is psychiatric 

No Reference Reference Reference Reference 

Yes 0.96 0.61 to 1.51 sa” 1.66 to 2.93 


RRR, relative risk ratio. 


a Reflects the presence of any of the listed diagnoses or RFVs; young adults served as the reference group. 


* P< 05. 


United States. The weights also 
account for nonresponse by EDs 
(those that did not participate in the 
NHAMCS survey), which averaged 
11.9% over the 5-study year period.’ 


6 to 11 y 


———- 12t017y 
18 to 24 y 


Data for the NHAMCS are collected 
during a randomly selected 4-week 
period of the year. Information from 
patient medical records is abstracted 
by trained staff members who use 
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standardized forms. All NHAMCS data 
are deidentified and publicly 
available, which allows for this study 
to be exempt from the local 
institutional review board. Beyond 
the NHAMCS, the only other data used 
in this study were retrieved from the 
US Census Bureau.*® Census data 
(from 2011 to 2015) were used for 
calculating population-based rates of 
psychiatric ED visits per 1000 youth 
in the United States. 


Variables 
Identification of Psychiatric ED Visits 


The primary study outcome was 
probability of a psychiatric ED visit. 
These visits were identified on the 
basis of the following: (1) diagnoses 
classified by using the International 
Classification of Diseases, Ninth 
Revision codes*” and (2) reason for 
visit (RFV) (record of the patient’s 
reason for visiting the ED). 


Psychiatric ED visits were identified 
when any of the 3 RFV or ICD-9 codes 
were used to identify a psychiatric 
disorder, psychiatric RFV, or a mental 
health procedure. Psychiatric 
diagnoses, RFVs, and procedures 
were identified by using 
preestablished criteria by researchers 
from the National Center for Health 
Statistics (NCHS).*° Visits were 
classified as mood (ie, depression, 
mania and/or hypomania, or anxiety), 
behavioral (ie, aggression or conduct 
problems), substance use disorder 
(SUD), psychosis, or other psychiatric 
reasons (ie, personality disorder or 
a mental health procedure). For 
suicide attempt and intentional self- 
harm, these visits were identified 
separately by using RFV (5818.0, 
5820.0, and 5820.1) and V codes 
(V62.84).*° 


Demographic Characteristics 


Demographic variables related to the 
child included age, US region, sex 
(male or female) and race and/or 
ethnicity (a 4-level variable created 
by the NCHS; see Table 1 for details). 
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TABLE 3 Race- and Ethnicity-Related Characteristics of Psychiatric ED Visits Among Youth in the Visit Characteristics 


United States a 
a: A host of visit-related descriptors 


Non-Hispanic Non-Hispanic African Hispanic Non-Hispanic were available. These included 


White American Other ; ; 
insurance type, time of day, day of 
Count of total visits (weighted) 7990/2 719 50 959 526 27 564 070 5 874 405 visit, whether a mental health 
Count of psychiatric ED visits 8 162 101 2 607 569 2 074122 297 994 ARN | dui h 
(weighted) professional was seen during the 
Estimated youth in the United 217 594589 56413412 92679177 36265765 visit, whether the patient arrived via 
States, 2011-2015 ambulance, and urgency of visit 
Psychiatric ED visits, % of all visits (count) (dichotomized as urgent [emergent, 
aca [semiurgent and not urgent]). Length 
2012 9.3 (1499311) 4.9 (342599) 6.5 7.1 (61684) - 5 5 5 
(411 193) of visit was grouped in quartiles. 
2013 9.8 (1576 000) 5.9 (387 165) 6.1 7.9 (60 165) Disposition or whether the visit 
(339 747) resulted in a hospitalization was 
tie a medical unit, admission to 
2015 10.3 (1774709) 10.8 (795 382) 9.1 8.9 (79543) ania 
(590 101) a psychiatric unit, or other (eg, 
Psychiatric ED visits per 1000 US youth discharge from the hospital). See 
2011 37.9 46.1 16.7 79 Table 1 for details. 
2012 $4.5 50.4 22.2 8.0 
2013 56.1 54.3 18.3 8.5 
2014 38.5 49.7 22.8 5.4 Missing Data 
2015 40.8 70.6 31.9 11.0 Do p 
Region,” % of all psychiatric ED visits Prevalent missing data (>5%) were 
Northeast 19.8 19.4 21.1 24.1 observed for urgency of visit (22%), 
Midwest 28.1 26.8 11.8 19.3 race and/or ethnicity (21%), and 
South 52.7 44.0 24.1 24.8 insurance type (8%). Imputation 
West 19.4 9.8 45.1 51.8 
ae procedures, conducted by the 
6-11 56 79 66 136 NCHS,°° were used for race and/or 
{0-17 979 98 9 37.1 276 ethnicity only. In the multivariate 
18-24 66.9 63.9 06.3 08.8 analyses, a missing data indicator was 
is % , a ie ae ie included for each of these variables 
emale : ‘ f . D 
are a oe a Sa (excluding race and/or ethnicity) for 
mume 07 the purposes of retaining the full 
Private 41.7 18.6 22.6 40.4 sample. 
Public Of.o 04.5 06.0 50.6 
Self-pay 15.1 19.7 14.6 9.4 
Other 5.7 7.4 6.8 14.5 Analysis 
Visit on a weekend, % Trends in psychiatric ED visits are 
No 13.4 74.5 76.8 69.1 psy , 
Ves 96.6 955 03.9 20.9 reported as proportions (number of 
Arrival time, % psychiatric ED visits over total ED 
Morning (<11 am) 29.4 26.6 25.2 27.4 visits) and counts by using the 
Afternoon (11 aM—4 pm) 28.7 51.5 52.7 26.9 NHAMCS data. Rates (visits per 1000 
gent a ae me a =e youth in the United States) and ratios 
No 083 375 97.8 39 4 of rates (reflecting percent change) 
Yes 71.7 62.4 we. 67.6 were calculated by using the census 
Length of visit, % data. To address the first study 
<90 min 20.9 20.5 17.5 20.7 question, a logistic regression model 
ae aoe oe a e was used to examine changes in the 
180-359 min 28.3 28.4 35.6 27.9 ae Bf 
=G h 902 959 994 99 9 odds of a psychiatric ED visit over 
Arrive by ambulance, % time among youth 6 to 24 years of 
No 24.0 28.2 27.4 36.0 age. For the second study question, 
Yes 16.0 11.8 12.9 64.0 separate logistic models were used to 
Mee Nee O eel ae test whether there was an increase in 
No 82.7 84.6 84.5 92.1 


the odds of a psychiatric ED visit over 
time for each individual age, sex, and 
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TABLE 3 Continued 


Non-Hispanic 
White 
Yes 17.3 
Admit to hospital, % 
No 80.5 
Yes or transfer to another 9.8 
hospital 
Transfer or admit to 9.7 
a psychiatric facility 
Psychiatric diagnoses or RFV“? 
% 
Mood 04.0 
Behavior 11.9 
SUD" 43.7 
Psychosis 6.2 
Suicide attempt or self-injury 12.9 
Other 29.7 
Count of psychiatric 1.45 
diagnoses or RFVs, mean. 
Primary diagnosis is 
psychiatric, % 
No 52.8 
Yes 67.2 


a Reflects the presence of any of the listed diagnoses or RFVs. 


* P< 05. 


racial and/or ethnic group (for a total 
of 9 separate models). 


For the third study question, a series 
of regression models were employed 
to identify differences in the 
demographic and visit-related factors 
associated with age, sex, and race 
and/or ethnicity. These analyses were 
limited to psychiatric ED visits only, 
with the group (age, sex, or race) 
serving as the outcome variable. For 
sex, a logistic regression model was 
employed, whereas a multinomial 
regression model was used for age 
and race and/or ethnicity because 
there were multiple discreet 
categories. As an example, the relative 
risk ratio for evening (from the 
multinomial model; see Table 2) can 
be interpreted as the relative 
probability of a psychiatric visit 
occurring in the evening (rather than 
the morning), is 2.35 times more 
likely to occur among children 
compared with young adults. 


All models used to assess predictors 
of psychiatric ED visits included year, 
region, sex, arrival time, weekend 
(versus weekday), and insurance type 
as covariates. All other variables 


Non-Hispanic African Hispanic Non-Hispanic 

American Other 
15.4 15.9 8.0 
81.8 84.0 80.5 
7.6 6.9 13.0 
10.6 9.9 6.8 
47.4 01.6 63.4 
18.6 16.9 6.7 
57.1 59.6 24.1 
9.5 9.7 2.1 
11.4 12.8 20.2 
535.0 27.5 56.4 
1.45 1.37 1.55 
50.2 57.9 Of.f 
64.8 62.1 62.3 


(eg, arrival by ambulance, urgency, 
length of visit, and diagnosis) were 
examined sequentially in individual 
models to avoid problems with 
multicollinearity. All regression 
models were examined at the visit 
level (by using the NHAMCS) among 
those with and/or without 
psychiatric needs. Whole population 
data from the census were not 
included in the models. All analyses 
were conducted in Stata 15.0 (Stata 
Corp, College Station, TX) by using the 
survey sampling weights. 


RESULTS 


Overall Trends in Psychiatric ED 
Visits 

Table 1 displays the 5-year trends in 
psychiatric ED visits by age group. 
Overall, there was a significant 
increase in the odds of a visit over 
time (odds ratio [OR]: 1.07; 95% 
confidence interval [CI]: 1.01 to 1.13; 
P = .01). Shown in Fig 1 and Table 1, 
increasing rates of psychiatric ED 
visits among youth in the United 
States (from 31.3 to 40.2 per 1000; 
a 28% increase) was also supported. 


Age Trends in Psychiatric ED Visits 


For children, there was no significant 
change over time in the probability of 
a psychiatric ED visit (OR: 1.05; 95% 
CI: 0.92 to 1.21; P = .44). There was 
a rise (5.2-8.0 psychiatric ED visits 
per 1000 US children; a 53% 
increase) by using whole population 
data as the denominator, which do 
not take into account the increase in 
ED visits overall. 


For adolescents, psychiatric ED visits 
were much more likely than for 
children (OR: 4.62; 95% CI: 3.73 to 
5.72; P < .001). Young adults were 
also much more likely to visit the ED 
for psychiatric purposes compared 
with children (OR: 4.93; 95% CI: 3.99 
to 6.10; P < .001), although there was 
no difference when compared with 
adolescents (OR: 1.07; 95% CI: 0.92 
to 1.23; P = .37). Notably, young 
adults had the highest overall 
population rates. 


The probability of a psychiatric ED 
visit significantly increased over time 
for adolescents (OR: 1.11; 95% CI: 
1.02 to 1.21; P = .01) and young 
adults (OR: 1.08; 95% CI: 1.01 to 
1.15; P = .03). By using census data, 
a larger increase was observed for 
adolescents (28.9-44.6 psychiatric 
ED visits per 1000 youth in the 
United States; a 54% increase) than 
for young adults (54.0-62.4 per 1000 
youth in the United States; a 15% 
increase). See Table 1 for details. 


Race and/or Ethnicity Trends in 
Psychiatric ED Visits 


Table 3 reveals trends in psychiatric 
ED visits across racial and ethnic 
groups. No significant change in the 
odds of a psychiatric ED visit was 
observed over time for the white 
group (OR: 1.03; 95% CI: 0.97 to 1.10; 
P = .31). However, a significant 
increase was observed among those 
in the African American (OR: 1.16; 
95% CI: 1.03 to 1.30; P = .01) and 
Hispanic groups (OR: 1.12; 95% CI: 
1.01 to 1.24; P = .02). Both of these 
findings remained significant even 
after adjusting for insurance status. 
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TABLE 4 Sex-Related Characteristics of Psychiatric ED Visits Among Youth in the United States No difference was found among the 
Female Sex Male Sex “other” race group (OR: 1.06, 95% CI: 
0.84 to 1.35, P = .59). The census data 


Count of total visits (weighted) 87 687 560 68 404 546 
Count of psychiatric ED visits (weighted) 6 582 345 6559 241 supported increasing rates among 
Estimated youth in the United States, 2011-2015 196 847 905 206 105 038 African American (46.1-70.6 per 
Psychiatric ED visits , % of all visits (count) 1000 youth, a 53% increase) and 
2011 6.9 (1 259 925) 8.9 (1 256 081) Hispanic patients (16.7-31.9, a 91% 
2012 6.9 (1 184 067) 8.6 (1 130 720) i h th t 
2013 6.7 (1122287) 10.0 (1.240 790) IIETe Se etede INESE TAES 
2014 7.4 (1320 195) 9.4 (1387 786) remained flat among non-Hispanic 
2015 9.5 (1695 871) 11.0 (1543 864) white patients (37.5-40.8; a 9% 
Psychiatric ED visits per 1000 US youth increase) and rare among all in the 
2011 52.0 ve “other” racial category (7.9-11.0; 
2012 50.1 21.4 39% j 
2013 28.4 30.0 a 39% increase). 

2014 33.4 33.6 = 
9015 431 375 Sex Trends in Psychiatric ED Visits 
ate all psychiatric ED visits ie He Table 4 reveals trends in psychiatric 

ortneéas ; : Ta ipi 
Haned oA 8 A ED visits stratified by sex. anere was 
South 378 999 a slight increase in the probability 
West 18.8 24.5 over time for female patients (OR: 
Youth, y, % 1.08; 95% CI: 1.01 to 1.16; P = .02). 
aoe a = No significant time trends were 
19-54 63 0 65 A observed for male patients (OR: 1.06; 
Race, % 95% CI: 0.99 to 1.13; P = .08). The 
Non-Hispanic white 64.2 60.0 probability of a psychiatric ED visit 
Non-Hispanic African American 18.7 20.9 was increased in male patients 
ee DEM a oe compared with female patients (OR: 
Oon-hispanic otner . f 
Bi 1.30; 95% CI: 1.17 to 1.45). Rates per 
Insurance, % i ; 
private 3A | 345 1000 youth also increased slightly 
Public 46.4 40.6 more for female patients (32.0-43.1; 
Self-pay 14.9 17.0 a 34% increase) than for male 
_ Other 0.0 ng patients (30.5-37.5; a 23% increase). 
Visit on a weekend, % 
No 75.4 74.6 2 = 
es ae ey Correlates of Psychiatric ED Visits 
Arrival time, % Age Group 
Morning (<11 am) 28.2 28.1 f i: S 
A econ eE 316 08.1 Child and visit-related characteristics 
Evening (5 pM—12 Am) 40.2 43.8 of psychiatric ED visits across age 
Urgent, % groups are shown in Table 1. In the 
No 51.7 28.4 multivariate analyses (Table 2), 
MeS E pia a children and adolescents were both 
Length of visit, % | likel i If . 
en 008 199 ess likely to have self-pay insurance, 
90-179 min 98.4 985 arrive by ambulance, and have an 
180-359 min 29.5 29.4 SUD diagnosis compared with young 
=6 h 21.2 22.2 adults. Children visiting the ED 
i (0) 
ane a were also less likely to live in the 
No 20.4 20.9 Ww dh ie h kend 
Ves 746 741 est an ave a visit on the weeken 
Mental health provider seen, % compared with young adults. Both 
No 82.5 84.5 children and adolescents were more 
Yes 17.4 15.4 likely to have an evening visit; and 
oa 
HOMIE TONES plat 20 adolescents only were more likely to 
No 81.9 80.7 h f _— T like] 
Yes or transfer to another hospital 74 10.4 ave ana P angeae Sey 
Transfer or admit to a psychiatric facility 10.7 8.9 to have the “other” insurance type (all 
Psychiatric diagnosis or RFV, * % P < .05). 
Mood 59.1 45.9 


Adolescents who visited the ED were 
more likely to be Hispanic, have an 
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TABLE 4 Continued 


Behavior 

SUD 

Psychosis. 

Suicide attempt or self-injury 

Other 

Count of psychiatric diagnoses or RFVs, mean 
Primary diagnosis is psychiatric, % 

No 

Yes 


a Reflects the presence of any of the listed diagnoses or RFVs 


“P< 05 


urgent visit, see a mental health 
provider, and be transferred or 
admitted to a psychiatric facility 
compared with young adults. In terms 
of diagnoses, adolescents were more 
likely to have a primary psychiatric 
diagnosis related to the visit along 
with an increasing number of 
diagnoses. Adolescents were also 
more likely to have a mood, 
behavioral, suicide, and other 
diagnosis or RFV compared with 
young adults (all: P < .05). Children 
were far more likely to have 

a behavioral diagnosis or RFV and 
slightly more likely to have an “other” 
diagnosis. See Table 2 for details. 


Race and/or Ethnicity 


Characteristics of psychiatric ED 
visits across racial and/or ethnic 
groups are shown in Table 3. In the 
multivariate analyses (Table 5), white 
patients served as the reference 
group. African American patients 
were less likely to have a visit in the 
Western region of the United States, 
have a visit that was deemed 

urgent, and present with an SUD. 
They were more likely to have an 
insurance type other than private and 
have a behavioral problem coded as 
a diagnosis or RFV. Hispanic patients 
were less likely to have a visit occur 
in the Midwest and had a fewer 
number of diagnoses coded for their 
visit, whereas they were more likely 
to have a visit in the Western part of 
the United States and have public or 
self-pay insurance. The only 
significant difference for the “other” 


Female Sex Male Sex 
12.1 15.3 
57.1 44.2 
5.1 8.2 
14.6 10.9 
50.1 50.5 
1.44 1.44 
55.1 55.4 
64.9 66.6 


racial group was that they were less 
likely to have an SUD associated with 
their visit (all: P < .05). 


Sex 


Differences between male and female 
patients, in terms of demographic and 
visit-related characteristics of 
psychiatric ED visits, are shown in 
Table 4. In the multivariate models 
(Table 6), male patients were more 
likely to be younger and visit the 

ED for an SUD or psychosis compared 
with female patients. On the other 
hand, psychiatric ED visits were 

less likely to occur in the South 
among male patients. 


Suicidal-Attempt or Self-Injurious Visit 


Changes in the proportion of suicide- 
related visits (overall and within 
psychiatric visits) over time are 
shown in Table 7; population-based 
rates are also provided. Overall, there 
was an increase in the probability of 
a suicidal, self-injurious visit among 
adolescents only (OR: 1.27; 95% CI: 
1.10 to 1.48; P < .01); there was no 
increase over time for children or 
young adults (both: P > .80). There 
was also no significant increase in the 
probability of a suicide-related visit 
for female (OR: 1.11; 95% CI: 0.97 to 
1.27) or male patients (OR: 1.15; 95% 
CI: 0.96 to 1.37) when examined 
separately. When whole population 
rates were employed, similar 
increases were observed among 
female (4.4-6.7 per 1000 US youth; 
a 52% increase) and male patients 
(3.2-5.1; a 59% increase). Racial and/ 


or ethnic trends were not examined 
because of small sample sizes. 


DISCUSSION 


In the current study, we found that 
psychiatric visits to the ED among 
youth 6 to 24 years of age are 
increasing across the United States. 
The largest rise in such visits were 
found among adolescents and youth 
of color. It should be noted that the 
findings were heavily driven by 
2015, in which the largest increase in 
visits was observed. Nevertheless, 
these data reflect a continuation 

of historical trends in pediatric 
psychiatric ED visits’**° and are in 
line with recent increases in inpatient 
psychiatric hospitalizations.” 
Ultimately, it is unclear if the findings 
represent a change in identification 
(by providers) or reporting (by 
patients or family members) of 
mental health in the ED, a shift in the 
epidemiology of psychiatric disorders 
in the United States, or fluctuations in 
referral patterns or service-seeking 
behavior. 


This study unmistakably reveals that 
adolescents are a population with 
urgent mental health needs. Not only 
were their visits the most acute, but 
their probability of suicidal attempt 
and/or self-harm increased as well. 
The latter is consistent with the 
national trends in suicide ideation.** 
Surprisingly, only 37% of visits 
involving adolescents who visited the 
ED for a suicidal attempt and/or self- 
injury resulted in a mental health 
provider being seen. In the same vein, 
only 16% of all psychiatric ED visits 
resulted in a mental health provider 
being seen. This is not the first study 
to identify this gap (for suicide- 
related or general psychiatric ED 
visits), which is likely due to the 
limited access of mental health 
providers in the ED.*° With the 
growing presence of mental health 
in this setting, coupled with the 
length of these visits, this finding 
represents a missed opportunity for 


Downloaded from www.aappublications.org/news by guest on March 18, 2019 


KALB et al 


TABLE 5 Multivariate Analysis of Correlates of Psychiatric ED Visits Among Racial and Ethnic Groups 


African American 


RRR 

Year 114° 
Age group, y 

6-11 Reference 

12-17 0.86 

18—24 0.77 
Region 

Northeast Reference 

Midwest 0.95 

South 1.45 

West 0.49° 
Sex 

Female Reference 

Male 1.26 
Insurance 

Private Reference 

Public 3.36 

Self-pay 2.99" 

Other 293 
Visit on a weekend 

No Reference 

Yes 1.01 
Arrival time 

Morning (<11 am) Reference 

Afternoon (11 am—4 pm) 1.00 

Evening (5 PpM—12 am) 1.01 
Urgent 

No Reference 

Yes 0.67% 

Length of visit 1.15 
Arrive by ambulance 

No Reference 

Yes 1.50 
Mental health provider seen 

No Reference 

Yes 0.87 
Admit to hospital 

No Reference 

Yes or transfer 0.80 

Psychiatric facility 0.97 
Psychiatric diagnosis or RFV° 

Mood 0.77 

Behavior 1.60 

SUDs 0.74 

Psychosis 1.38 

Suicide attempt or self-injury 0.91 

Other 1.17 
Count of diagnoses and/or RFVs 0.94 

Primary diagnosis is psychiatric 0.95 


RRR, relative risk ratio. 


95% Cl 
1.03 to 1.27 


Reference 
0.50 to 1.46 
0.49 to 1.21 


Reference 
0.52 to 1.69 
0.85 to 2.50 
0.26 to 0.95 


Reference 
0.99 to 1.61 


Reference 
2.25 to 9.05 
1.81 to 4.92 
1.56 to 5.50 


Reference 
0.72 to 1.42 


Reference 
0.69 to 1.44 
0.73 to 1.42 


Reference 
0.45 to 0.97 
0.99 to 1.28 


Reference 
0.94 to 1.80 


Reference 
0.61 to 1.23 


Reference 
0.52 to 1.24 
0.58 to 1.63 


0.58 to 1.02 
1.09 to 2.34 
0.56 to 0.97 
0.89 to 2.50 
0.56 to 1.42 
0.87 to 1.97 
0.78 to 1.15 
0.69 to 1.27 


RRR 
1.10 


Reference 
1.57 
0.78 


Reference 
0.35. 
0.73 
DAT 


Reference 
1.15 


Reference 
O79. 
9.42 
1.84 


Reference 
0.87 


Reference 
1.13 
1.06 


Reference 
0.89 
1.08 


Reference 
1.23 


Reference 
0.81 


Reference 
0.69 
0.69 


0.91 
1.34 
0.73 
0.78 
0.82 
0.80 
0.75 
0.76 


Hispanic Other 
95% Cl RRR 95% Cl 
0.98 to 1.24 1.00 0.77 to 1.30 
Reference Reference Reference 
0.68 to 2.76 0.43 0.08 to 2.40 
0.38 to 1.58 0.35 0.07 to 1.73 
Reference Reference Reference 
0.20 to 0.61 0.59 0.15 to 2.39 
0.41 to 1.27 0.65 0.25 to 1.77 
1.31 to 3.41 1.55 0.66 to 2.81 
Reference Reference Reference 
0.85 to 1.59 1.08 0.58 to 2.01 
Reference Reference Reference 
1.85 to 4.00 0.97 0.55 to 2.72 
1.48 to 3.93 0.77 0.28 to 2.07 
0.91 to 5.02 2.68 0.98 to 4.53 
Reference Reference Reference 
0.59 to 1.26 1.31 0.65 to 2.63 
Reference Reference Reference 
0.75 to 1.77 0.97 0.41 to 2.31 
0.72 to 1.56 1.10 0.55 to 2.31 
Reference Reference Reference 
0.39 to 2.01 0.94 0.41 to 2.14 
0.96 to 1.21 0.98 0.71 to 1.36 
Reference Reference Reference 
0.85 to 1.75 1.88 0.85 to 4.16 
Reference Reference Reference 
0.55 to 1.19 0.38 0.13 to 1.13 
Reference Reference Reference 
0.40 to 1.20 1.55 0.50 to 5.64 
0.42 to 1.11 0.66 0.22 to 1.96 
0.67 to 1.22 1.62 0.75 to 5.64 
0.87 to 2.06 0.41 0.14 to 1.25 
0.52 to 1.01 0.39" 0.20 to 0.69 
0.44 to 1.41 0.42 0.15 to 1.15 
0.55 to 1.23 1.82 0.88 to 5.74 
0.55 to 1.16 1.55 0.68 to 2.62 
0.60 to 0.94 0.80 0.57 to 1.71 
0.52 to 1.11 1.44 —0.01 to 2.5 


a Reflects the presence of any of the listed diagnoses or RFVs; non-Hispanic white patients served as the reference group. 


* P< 05. 


screening and treatment planning. 
Nevertheless, there are promising 
efforts to increase the availability of 
expertise within the ED, notably 
through the use of telemedicine or 
other mobile health technologies and 
linkages to treatment afterward.**** 


initiatives. 


Data from this study support the 
continued funding and effort of these 


The rise in psychiatric ED visits 
among youth of color is an important 
finding from this study. Again, 


this finding was heavily driven by 
2015. Although there were no visit- 
related differences (eg, time of day, 
arrival type, and urgency), there 
were several known demographic 
(eg, insurance type and geographic 
location) and diagnostic (eg, SUDs) 
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TABLE 6 Multivariate Analysis of Correlates of Psychiatric ED Visits Across Youth Sex 


Year 
Age group, y 
6-11 
12-17 
18—24 
Region 
Northeast 
Midwest 
South 
West 
Race and/or ethnicity 
White, non-Hispanic 
African American, non-Hispanic 
Hispanic 
Other 
Insurance 
Private 
Public 
Self-pay 
Other 
Visit on a weekend 
No 
Yes 
Arrival time 
Morning (<11 am) 
Afternoon (11 AM—4 pm) 
Evening (5 PM—12 am) 
Urgent 
No 
Yes 
Length of visit 
Arrive by ambulance 
No 
Yes 
Mental health provider seen 
No 
Yes 
Admit to hospital 
No 
Yes or transfer to another hospital 
Transfer or admit to a psychiatric facility 
Psychiatric diagnosis or RFV 
Mood 
Behavior 
SUDs 
Psychosis 
Suicide attempt or intentional self-injury 
Other 
Count of diagnoses and/or RFVs 
Primary diagnosis is psychiatric 
No 
Yes 


* P< 05 


Female Sex Male Sex 
OR 95% Cl 

Reference 0.97 0.91 to 1.05 
Reference Reference Reference 
Reference 0.29 0.18 to 0.49 
Reference 0.33" 0.54 to 0.60 
Reference Reference Reference 
Reference 0.95 0.74 to 1.21 
Reference 0.67. 0.51 to 0.87 
Reference 1.20 0.92 to 1.80 
Reference Reference Reference 
Reference 1.26 0.99 to 1.61 
Reference 1.14 0.83 to 1.58 
Reference 1.09 0.58 to 2.05 
Reference Reference Reference 
Reference 0.81 0.62 to 1.06 
Reference 1.17 0.78 to 1.78 
Reference 1.55 0.99 to 2.40 
Reference Reference Reference 
Reference 0.94 0.77 to 1.67 
Reference Reference Reference 
Reference 0.89 0.67 to 1.19 
Reference 1.09 0.84 to 1.42 
Reference Reference Reference 
Reference 1.25 0.97 to 1.61 
Reference 1.00 0.99 to 1.00 
Reference Reference Reference 
Reference 0.99 0.79 to 1.22 
Reference Reference Reference 
Reference 0.89 0.70 to 1.15 
Reference Reference Reference 
Reference 1.54 1.08 to 2.20 
Reference 0.90 0.65 to 1.25 
Reference 0.60 0.48 to 75 
Reference 1.28 0.91 to 1.78 
Reference 141 1.13 to 1.75 
Reference 1.70 1.11 to 2.61 
Reference 0.79 0.52 to 1.06 
Reference 1.02 0.79 to 1.33 
Reference 1.03 0.89 to 1.20 
Reference Reference Reference 
Reference 1.14 0.91 to 1.43 


differences between racial and 
ethnic groups.” Future research is 
needed to unpack these findings, 
particularly among African American 
youth who had less urgent visits. 


A few sex-related differences were 
identified in this study. Although 
there was no difference in time 
trends, male patients were slightly 
more likely to visit the ED for 


psychiatric reasons, which departs 
from previous pediatric research.” 
This discrepancy is likely due to 

the inclusion of young adults, who 
had the highest proportion of 
substance use-related ED visits. In 
this study and others, male patients 
are known to have increased rates of 
substance abuse compared with 
female patients.*° Surprisingly, female 
patients were not significantly more 
likely to visit the ED for suicide or self- 
injury, a finding that may be partly 
due to the small samples sizes given 
the rarity of these visits.*° This finding 
supports the need for suicide 
prevention and interventions efforts 
among male patients as well as female 
patients, who could be overlooked. 


Although many researchers and 
clinicians look to the outpatient 
mental health system to prevent 
psychiatric ED visits, there is mixed 
evidence to suggest that a connection 
to mental health providers can reduce 
such visits.*”’® For instance, a recent 
systematic review revealed little 

to no evidence that engagement in 
outpatient, primary care, or 
community and/or school-based 
mental health services impacted ED 
use.?” One explanation may be that 
providers themselves may be sending 
children to the ED during times of 
crisis, as reflected by a recent national 
survey that revealed that 60% of child 
and adolescent psychiatrists 
recommended that parents take their 
child to the ED during times of crisis.” 
Although the ultimate goal is to link 
youth with appropriate sources of care 
outside of the ED (particularly because 
these visits have been linked to 
iatrogenic outcomes,” poor follow- 
up,” and repeat visits**), the ED will 
likely have to play a continued role in 
treating both acute and chronic mental 
health issues in the future. As such, 
this study supports the continued 
effort to outfit EDs with the expertise 
to identify and treat mental health 
problems, to investigate novel 
approaches to crisis intervention (eg, 
mobile crisis centers, peer crisis 
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TABLE 7 Trends in ED Visits Related to Suicidal Attempts and/or Self-Injury important implications to both 


2011 2012 2013 2014 2015 Overall policy and practice. 
All ED visits, % 
6-11 y <0.1 <0.1 0.2 0.1 <0.1 <0.1 CONCLUSIONS 
12-17 y 1.4 13 2.7 a 3.5 2.2 a Ea 
18-24 y 13 1.0 0.7 0.8 11 0.9 Psychiatric ED visits among young 
Female patients 1.0 1.0 1.1 1.0 15 1.1 people increased nationally, most 
Male patients 0.9 0.7 1.1 1.1 1.9 1.0 notably in 2015. The largest 
Overall B 0.9 0.8 i Me E Le increases were observed among 
Psychiatric visits only, % do] t d th of col 
6-11 y 4.2 24 3 53 i7 4.2 E RE DERS 
12417 y" 15.8 15.5 28.7 24.8 26.2 227 whereas young adults had the 
18-24 y 10.9 9.9 7.1 7.6 9.1 8.9 highest overall population rates. 
Female patients 15.6 15.8 15.9 15.9 19.9 14.6 Visits related to suicidal attempt and 
Male patients 10.4 79 11.0 10.6 15.5 10.9 self-injury also increased among 
Overall 12.0 10.9 15.5 12.2 14.5 12.7 dol ts. Th dat tth 
Visits per 1000 US youth aa a a 2 emanate 
6-119 0.2 0.1 0.5 0.4 0.1 0.3 continued funding and deployment 
12-17 y 4.6 3.7 li 15 11.7 7.1 of evidenced-based interventions 
18-24 y 0.9 0.0 5.9 4.5 9.7 4.8 that can effectively reach families 
Female patients 4.4 4.1 45 46 6.7 49 and young adults during times of 
Male patients 3.2 2.2 3.5 5.6 5.1 5.0 E 
Overall 3.7 3.2 3.9 4.1 5.8 4.9 ease 
* P< 05. 


ABBREVIATIONS 


CI: confidence interval 

ED: emergency department 

NCHS: National Center for Health 
Statistics 


services, crisis stabilization beds, and 
telemedicine crisis services),**** or to 
design new models of urgent mental 
health care that are accessible 
(particularly in the evening). 


inability to confirm diagnostic 
validity is another limitation 
because consistent application of 
diagnostic criteria may 
systematically vary (across 


NHAMCS: National Hospital 


There are several study limitations 


that require mentioning. First, there 


were missing data across a few 


variables, namely visit acuity and race 
and/or ethnicity, although the latter 


was imputed by the NCHS.°° The 


providers and over time) in 
unknown ways. These limitations 
are offset by the nationally 
representative sample, in-depth 


analysis across numerous variables, 


and novel findings that have 


Ambulatory Medical 
Care Survey 

OR: odds ratio 

RFV: reason for visit 

SUD: substance use disorder 
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WIC Food Package Changes: Trends in 
Childhood Obesity Prevalence 
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Erica L. Kenney, ScD, MPH>* 


OBJECTIVES: To evaluate the association of the 2009 changes to the US Special Supplemental 
Nutrition Program for Women, Infants, and Children (WIC) food package and childhood 
obesity trends. We hypothesized that the food package change reduced obesity among 
children participating in WIC, a population that has been especially vulnerable to the 
childhood obesity epidemic. 


METHODS: We used an interrupted time-series design with repeated cross-sectional 
measurements of state-specific obesity prevalence among WIC-participating 2- to 4-year-old 
children from 2000 to 2014. We used multilevel linear regression models to estimate the 
trend in obesity prevalence for states before the WIC package revision and to test whether the 
trend in obesity prevalence changed after the 2009 WIC package revision, adjusting for 
changes in demographics. In a secondary analysis, we adjusted for changes in macrosomia and 
high prepregnancy BMI. 

RESULTS: Before the 2009 WIC food package change, the prevalence of obesity across states 
among 2- to 4-year-old WIC participants was increasing by 0.23 percentage points annually 
(95% confidence interval: 0.17 to 0.29; P < .001). After 2009, the trend was reversed (— 0.34 
percentage points per year; 95% confidence interval: —0.42 to —0.25; P < .001). Changes in 
sociodemographic and other obesity risk factors did not account for this change in the trend in 
obesity prevalence. 


CONCLUSIONS: The 2009 WIC food package change may have helped to reverse the rapid increase 
in obesity prevalence among WIC participants observed before the food package change. 


QO © 


“Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, Massachusetts; 
Departments of "Social and Behavioral Sciences and ‘Nutrition, Harvard TH. Chan School of Public Health, 
Harvard University, Boston, Massachusetts; “Department of Health Policy and Management, Mailman School of 
Public Health, Columbia University, New York, New York; “Department of Prevention and Community Health, Milken 
Institute School of Public Health, George Washington University, Washington, District of Columbia; and “New York 
Academy of Medicine, New York, New York 


Ms Daepp compiled the data, conducted the analyses, drafted components of the manuscript, and 
contributed to the conceptualization of the analytic approach; Dr Gortmaker conceptualized the 
study and contributed to the writing of the manuscript; Drs Long and Wang contributed to the 
conceptualization of the study and the writing and editing of the manuscript; Dr Kenney oversaw the 
analysis, contributed to the conceptualization of the analytic approach, and drafted components of 
the manuscript; and all authors approved the final manuscript as submitted. 


DOI: https://doi.org/10.1542/peds.2018-2841 
Accepted for publication Jan 29, 2019 


Address correspondence to Erica L. Kenney, ScD, MPH, Department of Nutrition, Harvard T.H. Chan 
School of Public Health, 665 Huntington Ave, Boston, MA 02115. E-mail: elk782@mail.harvard.edu 


Downloaded from www.aappublications.org/news by guest q 


PEDIATRICS Volume 143, number 5, May 2019:e20182841 


WHAT’S KNOWN ON THIS SUBJECT: Changes to the 
Special Supplemental Nutrition Program for Women, 
Infants and Children (WIC) in 2009 decreased intake of 
foods and beverages associated with excess weight 
gain. Surveillance data suggest obesity prevalence 
among WIC-participating children declined from 2010 
to 2014. 


WHAT THIS STUDY ADDS: We quantify the association 
of the WIC package change and childhood obesity 
using a quasi-experimental study design and 
accounting for state-level differences, race and/or 
ethnicity, poverty trends, and trends in macrosomia 
and high prepregnancy BMI among WIC participants. 


To cite: Daepp MIG, Gortmaker SL, Wang YC, et al. WIC 


Food Package Changes: Trends in Childhood Obesity 
Prevalence. Pediatrics. 2019;143(5):e20182841 


ARTICLE 


The Special Supplemental Nutrition 
Program for Women, Infants, and 
Children (WIC) provides vouchers for 
nutrient-dense foods and beverages 
to more than half of all US infants and 
one-quarter of low-income pregnant 
and postpartum women and children 
<5 years of age.” To be eligible, 
households must have an income 
<=185% of the federal poverty level 
or participate in several other federal 
aid programs and be at nutritional or 
medical risk. In addition to being at 
risk for poor nutritional status, WIC- 
eligible children have also been 
particularly vulnerable to the 
childhood obesity epidemic.* 


In 2009, the lists of foods that could 
be purchased with WIC vouchers 
(known as the WIC food packages), 
which includes basic food categories, 
including milk, cheese, beans, peanut 
butter, eggs, juice, cereal, and grains, 
were updated to better align with 
dietary guidelines. The new package 
provides extra cash allowances for 
fruits and vegetables, cuts the 
previous juice allowance in half, 
requires low-fat or skim milk for 2- to 
4-year-olds, reduces cheese, and 
requires whole-grain instead of 
refined-grain products (among other 
changes).” Evidence suggests the 
revised WIC package has beneficially 
impacted household diets, including 
an increase in Healthy Eating Index 
scores,’ reductions in juice purchases, 
and increases in the intake of fruit 
and whole grains.° A recent analysis 
of Nielsen Homescan Consumer Panel 
data found that WIC households’ 
purchases of total calories declined 
by 11% after the package change, 
whereas purchases of fruits and 
vegetables increased.’ 


On the basis of these well- 
documented improvements in 
nutrition after the WIC package 
change, the risk of childhood obesity 
among WIC-participating children 
may also have declined. The Centers 
for Disease Control and Prevention 
(CDC) reported a decline in the 
prevalence of obesity and severe 


obesity among WIC-participating 
children from 2010 to 2014 but did 
not directly test whether the WIC 
package change was associated with 
a change in childhood obesity trends.’ 
Given the variation in childhood 
obesity rates by state,’ evaluating 
longer-term trends in obesity 
prevalence and the potential 
differential impact of the WIC 
package change by state is critical. A 
focus on state-level trends is 
particularly important given the level 
of control that state WIC agencies 
have over program administration’; 
states had discretion for 
implementing certain aspects of the 
revised food packages and may have 
differed in the extent to which 
healthier foods were made 
available.*° In this article, we evaluate 
whether national and state-level 
trends in obesity prevalence among 
WIC-participating 2- to 4-year-old 
children changed after the 2009 WIC 
package revision, adjusting for 
sociodemographic composition and 
other risk factors for childhood 
obesity. 


METHODS 


Study Design 


This quasi-experimental study 
employed interrupted time-series 
analysis ’’ by using repeated cross- 
sectional measurements of obesity 
prevalence among WIC-participating 
children at the state level to test the 
potential impact of the 2009 WIC 
package change on childhood obesity 
trends. In interrupted time-series 
analysis, regression models estimate 
trends in population measures of an 
outcome before a prespecified time 
when a policy change (or other 
intervention) occurs and then test 
whether the trend changes 
afterward."* In this study, we 
compared population trends in 
obesity prevalence among WIC- 
participating 2- to 4-year-olds before 
and after 2009 to estimate whether 
there were changes in obesity 


prevalence or changes in the trend in 
obesity prevalence that may be 
attributable to the package change.” 
To reduce confounding from other 
concurrent trends, such as the Great 
Recession of 2007 to 2009, which 
occurred during a similar time period 
as the package change and which 
resulted in an increase in the use of 
federal food assistance and new 
populations using such assistance, we 
controlled for changes in racial-ethnic 
composition and child poverty at the 
state level. In an exploratory analysis 
including a subsample of states with 
complete data, we also examined 
whether changes in WIC-participating 
children’s birth weight and maternal 
BMI were confounders. 


Measures and Data Sources 


Our primary outcome is obesity 
prevalence, calculated as the 
percentage of 2- to 4-year-old 
children enrolled in the WIC in 

a given state or territory with sex- 
specific BMI for age =95th percentile 
on the 2000 CDC growth charts.’* We 
obtained the state-level prevalence of 
obesity among WIC-participating 2- 
to 4-year-olds in 2008, 2010, 2012, 
and 2014 from data.gov files** 
derived from the WIC Participant and 
Program Characteristics (WIC-PC), 

a national census of participants in 
the WIC administered every 2 years 
by the US Department of Agriculture's 
Food and Nutrition Service.” In 
addition, we used prevalence 
estimates from Pan et alt? for 2000, 
2004, 2010, and 2014. In 

a comparison of data for the 2 years 
of overlap (2010 and 2014), we find 
negligible differences between the 
data sets (Supplemental Information). 
We excluded Hawaii from our main 
analyses given concerns about data 
quality.® Our data set includes data 
from 49 states in 6 years, or 294 
observations; these data were 
collected from all 2- to 4-year-olds for 
whom obesity prevalence was 
determined, a group that ranged in 
number from 2 253 471 children in 
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2000 to 3 152 137 children in 2012 in 
total across all 49 geographies. 


To account for changes in the racial- 
ethnic composition of the WIC 
population, we calculated the 
percentage of 2- to 4-year-old WIC 
participants classified as white, 
African American, Asian American 
and/or Pacific Islander, American 
Indian, or Hispanic for each state and 
year. The WIC-PC reports for 2008, 
2010, 2012, and 2014 included state- 
level race-ethnic breakdowns for 
children between the ages of 2 and 
4 years but not for 2000 and 2004. 
We obtained data on race and 
ethnicity for 2000 and 2004 from the 
WIC data’®*’; when comparing these 
with the WIC-PC report data’* for the 
3 years of overlap (2008, 2010, and 
2012), we found high correlations 
between measures of racial and 
ethnic makeup (Supplemental 
Information). 


To account for possible shifts in child 
poverty, we assessed state-level child 
poverty rates with the Supplemental 
Poverty Measure (SPM) for children 
ages 0 to 17 years,” calculated by the 
Center on Poverty and Social Policy at 
Columbia University.’® The SPM is 

a posttax measure of poverty used by 
the Census Bureau and the Bureau of 
Labor Statistics that accounts for in- 
kind transfers and other benefits as 
well as nondiscretionary expenses, 
such as medical or child care costs." 
The SPM is considered an 
improvement over the official poverty 
measure in its use of a poverty 
threshold updated for contemporary 
expenditure patterns“°; we use 

a measure anchored to 
nondiscretionary spending (for food, 
clothing, shelter, and utilities) from 
2008 to 2012*° and adjusted for 
geographic differences in costs of 
living.’ 


For the exploratory analysis of 
whether changes in the 2- to 4-year- 
olds’ history of macrosomia or 
maternal prepregnancy weight status 
could have played a role in changes in 


childhood obesity prevalence among 
WIC-participating children, we 
extracted the proportions of WIC 
participants with a birth weight 
>4000 g and of WIC-participating 
mothers with a prepregnancy BMI 
classified as obese (>29.1, the cutoff 
used by WIC agencies before Revision 
10 of the Nutrition Risk Criteria)** 
from the WIC-PC reports. 1%161722-28 
To address the lag between the 
measurement of maternal 
prepregnancy BMI and obesity in our 
population of 2- to 4-year-old 
children, we include prepregnancy 
BMI with a 4-year lag. We note that 
only 41 states reported data on birth 
weight and prepregnancy BMI in at 
least 2 time points, and 45.1% of 
state-year observations are missing 
data for at least 1 of the 2 variables. 
We assessed the robustness of our 
results to the use of state-level birth 
weight data from the CDC Wonder 
Database natality public-use data,” 
but these data cover all children in 
the United States (rather than just the 
WIC population), and we were unable 
to identify a comparable data source 
for prepregnancy BMI. Because of the 
high incidence of missing data, we 
consider these analyses secondary 
and exploratory. 


Statistical Analyses 


We fitted segmented linear regression 
models to state-level obesity 
prevalence data as a function of year 
(centered at 2009), an indicator for 
post-WIC-package change (ie, 1 if 
year >2009 and 0 if otherwise), and 
an interaction term of the indicator 
and the number of years since the 
change. The coefficient for year can 
be interpreted as the annual 
percentage-point change in obesity 
per year on average across states, 
whereas the coefficient for the 
indicator of years after 2009 
represents the average percentage- 
point drop in obesity for all years 
after 2009 (a level change), and the 
coefficient of the interaction of the 
indicator and the year represents the 
average annual percentage-point 


change for each year after 2009 (a 
change in slope).’* All models were 
fitted with state fixed effects to 
control for time-invariant differences 
in obesity prevalence between states 
as well as with state random effects. 
Hausman tests”” fail to reject the null 
hypothesis of no difference in 
coefficient estimates from fixed- 
versus random-effect specifications; 
thus, we report results from the more 
efficient random-effect models. The 
full model specification can be found 
in the Supplemental Information. 


We first fit baseline models including 
just the year, level, and trend 
variables as well as the random 
effects. The level variable was not 
significant, suggesting there was no 
immediate drop in obesity prevalence 
after the year 2009; this variable was 
thus excluded from the models. We 
then fit models with controls for child 
poverty as well as racial-ethnic 
composition to assess whether 
changes in the demographics of WIC 
participants could explain changes 

in obesity level. Lastly, for our 
exploratory analysis of the potential 
impact of changes in trends in 
macrosomia and prepregnancy BMI 
on trends in childhood obesity 
prevalence in this population, we fit 
a model including the prevalence of 
macrosomia and high maternal 
prepregnancy BMI among those 
states with complete data for these 
variables. Because complete case 
analysis may produce biased 

results when data are not missing 
completely at random, we further 
used multiple imputation by chained 
equations to impute missing data.’ 
We imputed 20 data sets in which 
macrosomia and the percentage of 
mothers with high prepregnancy BMI 
were iteratively predicted with 
regression equations including all 
covariates used in our main analyses; 
the resultant model for this 
exploratory analysis was fit on each 
data set with results pooled to reflect 
the uncertainty in the predicted 
values.” 
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Station, TX). Two-sided P values <.05 


were considered statistically 


by using Stata 14 (Stata Corp, College 
significant. 


The main analyses were conducted in 
R 3.4.4. Fixed- and random-effects 
models were fitted with the plm 
package.”? Sensitivity analyses and 
multiple imputation were conducted 
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RESULTS 


Demographic characteristics of WIC 


participants from 2000 to 2014 are 
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presented in Table 1. The average 
obesity prevalence among WIC- 
participating 2- to 4-year-old children 
across the states was 12.92% in 


2000, rose to an average of 15.13% in 


2008, stayed roughly the same in 
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2010, and then decreased to 14.24% 


by 2014. Although the 
sociodemographic composition of the 


2- to 4-year-old WIC-participant 
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population changed somewhat over 
this period, we did not identify 


statistically significant differences in 
the state-level mean racial makeup or 


percentage of children in poverty. The 


average percentage of 2- to 4-year-old 


WIC participants born with 


2008 and then leveled off from 2008 


macrosomia declined from 2000 to 
to 2014. Meanwhile, the percentage 


of WIC-participating women who had 


a prepregnancy BMI >29.1 increased 


steadily from 2000 to 2014. 


Results from the crude interrupted 


time-series analysis show that before 
2009, national obesity prevalence 
among 2- to 4-year-old WIC 


participants was increasing 0.26 


percentage points annually (95% 


confidence interval [CI]: 0.22 to 0.31; 
P < .001; Table 2). After 2009, the 


trend was reduced by 0.55 


—0.43; P < .001). Observations are 

correlated within states, with ~78% 
of the variance being attributable to 
variation between states. Predictions 


percentage points (95% CI: —0.66 to 
for each of the 49 states with 


complete data are presented in Fig 1. 
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TABLE 2 Multivariate Random-Effects Models Predicting the State-Level Obesity Prevalence (Percentage 
Points) Among 2- to 4-Year-Old WIC Participants, 2000—2014 


Intercept (95% Cl) 
Annual trend’, yrs (95% Cl) 


15.90*** (14.93 to 16.17) 
0.26*** (0.22 to 0.31) 


Model 1 Model 2 


16.01*** (15.30 to 16.71) 
0.23" (0.17 to 0.29) 


Change in annual trend after 2009 (95% Cl) —0.55*** (—0.66 to —0.43) —0.57*** (—0.69 to —0.44) 


Race and/or ethnicity, %° (95% Cl) 
American Indian 
Asian American 
African American 
Hispanic 

Children in poverty (standardized)°, % 
(95% Cl) 

ICC 

N 

N groups 


— 0.07 (—0.003 to 0.14) 
— 0.19 (0.01 to 0.37) 

= 0.03 (—0.00 to 0.06) 
— 0.03 (—0.001 to 0.05) 
= 0.07 (—0.25 to 0.39) 


0.78 0.75 
294 294 
49 49 


Results are reported for models with state random effects; Durbin-Wu-Hausman tests fail to reject the null hypothesis of 
no significant differences in consistency between models with fixed and random effects. —, not applicable. 

a The annual trend is calculated with the year centered in 2009. 

b Demographic makeup is presented as the number of children with race reported in each category as a percentage of 
(for 2000 and 2004) all 1- to 4-y-old children enrolled in the WIC or (for 2008—2014) all 2- to 4-y-old WIC participants as 
a percentage of all children for whom a race in the 5 categories was reported. We further subtract the population- 
weighted mean demographic makeup from each of these variables so the intercept in model 2 can be interpreted as the 
mean overall obesity prevalence in 2009 for a state with a demographic makeup comparable to the demographic makeup 
of our population across all observations (1.14% American Indian, 3.24% Asian American and/or Pacific Islander, 19.95% 
African American, 44.16% Hispanic, and 31.5% white) and a child poverty rate equal to the average across state-year 


observations (14.84). 


c Standardized rate for children ages 0 to 17 y calculated with the anchored SPM available from the Center on Poverty and 


Social Policy at Columbia University for all US states.'9 
* P< 05; ** P < 001 


After adjusting for changes in racial- 
ethnic composition and child 
poverty, the results were essentially 
unchanged. We estimated a pre-2009 
annual trend of a 0.23 percentage- 
point increase in childhood obesity 
prevalence (95% CI: 0.17 to 0.29; 

P < .001) and a post-2009 change in 
slope of — 0.57 percentage points 
annually (95% CI: — 0.69 to — 0.44; 
P < .001). On the basis of these 
adjusted results, we estimated that 
after the package revisions in 2009, 
there was a decline in childhood 
obesity of 0.34 percentage points per 
year. The intraclass-correlation 
coefficient (ICC) was marginally 
reduced after controlling for 
sociodemographic variables (0.75), 
suggesting that between-state 
differences in sociodemographic 
makeup explained little of the 
between-state differences in obesity 
prevalence. 


Our exploratory analysis of the 
potential influence of changes in 
macrosomia and high maternal 


prepregnancy BMI on trends in 
childhood obesity prevalence 
suggests that, although controlling for 
macrosomia and prepregnancy BMI 
may have slightly attenuated the 
change in trend, these 2 factors have 
little contribution to the observed 
drop in childhood obesity prevalence 
after 2009 (Supplemental Tables 3 
and 4). 


DISCUSSION 


We found, using an interrupted time- 
series analysis, that the 2009 WIC 
package change was associated with 
a shift in the trend of childhood 
obesity prevalence among 2- to 4- 
year-old WIC participants from a 0.23 
-percentage-point annual increase to 
a 0.34 percentage-point annual 
decrease. Changes in the racial-ethnic 
and income composition of the WIC 
population did not explain this 
change in trend, nor did changes in 
maternal prepregnancy BMI or 
macrosomia prevalence. 


A change in the trend in obesity 
prevalence related to dietary 
changes resulting from the 2009 
package change is plausible. A 
substantial body of evidence has 
shown that the dietary habits of WIC 
participants improved from before to 
after the package change. 
Participating families purchased 
11% fewer total calories after the 
package change compared with 
before” and also improved overall 
diet quality in ways that may protect 
against obesity and facilitate 
healthier growth. Participants 
purchased and consumed less fruit 
juice, refined grains, grain-based 
desserts, and sugar-sweetened 
beverages while increasing 
purchases and consumption of fruits, 
vegetables, and whole grains.” ”’4 
This dietary pattern has been 
associated with less weight gain in 
both children and adults.2°°° Small, 
beneficial reductions in total energy 
intake could result in small 
population-wide reductions in 
average BMI.'°°”°8 


Although overall our analysis 
revealed declining obesity 
prevalence after the package change, 
our results also suggest variation 
between states that was not 
explained by differences in racial- 
ethnic makeup or child poverty. 
Although the difference may be due 
to unmeasured heterogeneity in WIC 
populations across states, another 
possibility is that differences exist in 
how effectively the WIC package 
changes were implemented in each 
state. For example, recent research 
documents disparities across states 
in minimum stocking requirements 
for WIC-participating vendors.*° 
Future researchers should evaluate 
whether differences in fidelity to the 
WIC package changes or other 
implementation factors may have 
played a role in potential differences 
in childhood obesity prevalence 
trends across the states. 


Our analysis is subject to a number 
of limitations. We lack individual- 
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FIGURE 1 


Year 


Obesity prevalence and prediction by state. Black points indicate the measured obesity prevalence among children ages 2 to 4 years in each year; lines 
show the fit from model 1 in Table 2, with red indicating the predicted trend before 2009 and blue indicating the trend after 2009. 


level BMI data over time as well as 
data on how energy intake and 
expenditure may have changed, 
which would provide a mechanistic 
explanation of our findings. Perhaps 
most importantly, we are unable to 


control for the possibility that some 
other event between 2008 and 2010 
may be driving the results. The Great 
Recession, for example, may have led 
to changes in the WIC population 
such that families who had 


previously had higher incomes and 
lower obesity risk were newly 
enrolling. However, we included 
controls for socioeconomic and 
demographic variables to account for 
these changes. Furthermore, we 
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might expect to see changes driven 
by the Great Recession manifest 
most strongly in states that were 
more strongly affected, but Fig 1 
does not reveal evidence of 
heterogeneity in the change in trend 
between the most strongly affected 
states (FL, NV, AZ, and CA) and states 
considered more “recession proof,’ 
such as North Dakota, South Dakota, 
Kansas, or Oklahoma.°” The Patient 
Protection and Affordable Care Act 
was another major policy change in 
2009, but existing evidence suggests 
no relationship between increased 
access to health insurance and 
obesity risk.*°** Although it has 
been suggested that the decline 
could be the result of other public 
health interventions, such as 
increased breastfeeding promotion 
in addition to that provided by WIC 
or obesity-prevention efforts in child 
care settings,’ this is unlikely. First, 
carefully controlled studies suggest 
no causal link between breastfeeding 
and childhood obesity.**~** Second, 
although there are promising child 
care initiatives to prevent obesity,” 
these were not widely disseminated 
in 2009 and thus could not logically 
have influenced obesity trends at 
that point. Similarly, although the 
Head Start program, which also 
reaches 2- to 4-year-olds in poverty, 
has been linked with potential 
reductions in obesity risk,*° there 
was no substantial change in Head 
Start in 2009 that could explain such 


changes in obesity trends. The 
inclusion of a control group of 
non-WIC participants would 
improve our ability to control for 
non-WIC-related policies, societal 
trends, or interventions that could 
have been impacting obesity in this 
population at the same time, but we 
are unable to identify a control 
group similar in age and 
socioeconomic characteristics to our 
sample because such a high 
proportion of low-income children 
participate in the WIC. Although the 
NHANES can provide estimates for 
this age group overall, the sample 
sizes for this age group are small 
once children participating in the 
WIC are excluded.” Comparing 
existing NHANES data on trends in 
childhood obesity prevalence with 
the WIC trends does not suggest that 
the overall trends in childhood 
obesity prevalence exactly mirrored 
the WIC trends. Although obesity 
prevalence leveled off between 2011 
and 2014 for children ages 6 to 

11 years, most of whom would have 
been too old to be affected by the 
WIC package change,*® changes in 
the prevalence of obesity for 2- to 5- 
year-olds were found to have 
declined from 2003 to 2004 and 
have had no clear trend since. As 
more years of data as well as data on 
state-level differences in program 
implementation become available, 
further research should continue to 
evaluate the effect by, for example, 
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examining heterogeneity in the 
change in trend on the basis of 
differences in WIC package changes 
by state. 


CONCLUSIONS 


This analysis demonstrates that the 
2009 WIC package was associated 
with a reversal of the increasing trend 
in obesity prevalence among WIC 
participants observed from 2000 

to 2014. 
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HPV Vaccine Delivery Practices by 
Primary Care Physicians 
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Laura P. Hurley, MD, MPH,®* Michaela Brtnikova, PhD, MPH,?° Brenda L. Beaty, MSPH,? Elissa Meites, MD, MPH,° 
Shannon Stokley, MPH,° Megan C. Lindley, MPH° 


BACKGROUND AND OBJECTIVES: To examine, among pediatricians and family physicians (FPs) (1) 
human papillomavirus (HPV) vaccine delivery practices, (2) delivery experiences, and (3) 
attitudes regarding new 2-dose HPV vaccination schedules. 


METHODS: We surveyed nationally representative networks of pediatricians and FPs by Internet 
or mail from July 2018 to September 2018. Multivariable regression was used to assess 
factors associated with refusal or deferral rates of =50% among 11- to 12-year-old patients. 


RESULTS: The response rate was 65% (302 pediatricians and 228 FPs included). Pediatricians 
who strongly recommended the HPV vaccine ranged from 99% for patients =15 years old 
(female) to 83% for those 11 to 12 years old (male); FPs ranged from 90% for patients 
=15 years old (female) to 66% for those 11 to 12 years old (male) (P < .0001 between 
specialties). Sixty-five percent of pediatricians and 42% of FPs always or almost always used 
presumptive style when discussing the HPV vaccine (P < .0001). Overall, 40% used standing 
orders and 42% had electronic alerts. Among pediatricians, the proportion reporting a refusal 
or deferral rate =50% was 19% for female patients and 23% for male patients 11 to 12 years 
old; FPs reported 27% and 36%, respectively. In the multivariable regression (both sexes), 
refusal or deferral was associated with physicians not strongly recommending the HPV 
vaccine to 11- to 12-year-old patients, not using a presumptive style, perceiving less 
resistance when introducing the HPV vaccine to a 13-year-old patient versus an 11- or 12- 
year-old patient, and anticipating an uncomfortable conversation when recommending the 
HPV vaccine to an 11- or 12-year-old patient. Eighty-nine percent of pediatricians and 79% of 
FPs reported that more adolescents <15 years old are completing the HPV series now that 
only 2 doses are recommended. 


CONCLUSIONS: Although most physicians strongly recommend the HPV vaccine to 11- to 12-year- 
old patients, our data reveal areas for improvement in recommendation and delivery methods. 
Most physicians perceive that the 2-dose schedule is resulting in higher HPV completion rates. 


WHAT’S KNOWN ON THIS SUBJECT: Although diseases caused by human 


a . . . . . . . . 
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for patients 11 to 12 years old, many are not using a presumptive style when 
introducing the HPV vaccine, standing orders, or electronic alerts for HPV delivery. 
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Disease caused by human 
papillomavirus (HPV) remains 

a major public health problem 
globally.” Nearly all cervical and 
anal cancers; 63% to 75% of vulvar, 
vaginal, and penile cancers; and 
~70%° of oropharyngeal cancers are 
attributable to HPV.“’~*° Annually in 
the United States, 33 700 new cancers 
are related to HPV, and 4175 women 
die of cervical cancer." 


Effective HPV vaccines have been 
routinely recommended at ages 11 to 
12 years by the Advisory Committee 
on Immunization Practices (ACIP) 
since 2006 for girls and since 2011 
for boys.’”'* The recommendations 
targeting those 11 to 12 years old 
were based in part on facilitating 
implementation because other 
vaccines are recommended at this 
age. In addition, HPV vaccines are 
most effective when given before any 
HPV exposure, and vaccination at this 
age reaches most persons before 
initiation of sexual activity. 


HPV vaccines have demonstrated high 
efficacy in preventing cervical 
precancers, other genital cancers, 
oropharyngeal cancers, and genital 
warts.” Healthy People 2020 goals 
include coverage of 80% for all 
vaccines routinely recommended for 
US adolescents.’” Although these 
goals have been met for other 
adolescent vaccines, in the 2017 
National Immunization Survey-Teen, 
HPV initiation among adolescents 13 
to 17 years old was 69% for girls and 
63% for boys, and series completion 
was only 53% for girls and 44% for 
boys. Low HPV vaccination is 
related to a variety of barriers 
originating from health care 
providers and parents or 

patients. ’-*° Because of the crucial 
role of provider recommendation in 
parental decisions to vaccinate,’ ~~” 
a great deal of research*’°°-°° and 
intervention efforts*’-*° have been 
focused on improving provider 
communication regarding HPV 
vaccination. 


Until 2016, completion of the HPV 
series was defined as 3 doses over 6 
months. In late 2016, the ACIP 
recommended a 2-dose schedule for 
adolescents who initiate the HPV 
vaccination series at ages 9 through 
14 years.?” In the 2-dose schedule, 
the first and second doses of the HPV 
vaccine should be administered at 
least 6 to 12 months apart.*° Three 
doses remain recommended for 
persons who initiate the series at 
ages 15 through 26 years and for 
persons who are 
immunocompromised. In the context 
of continued suboptimal vaccination 
rates and recent changes to 
recommendations, our objectives 
were to examine the following among 
nationally representative panels of 
pediatricians and family physicians 
(FPs): (1) current delivery and 
communication practices regarding 
the HPV vaccine; (2) attitudes toward 
and experiences with HPV vaccine 
delivery; (3) rates of refusal or 
deferral of the HPV vaccine; (4) 
perceived barriers to delivery of the 
HPV vaccine; and (5) knowledge, 
practices, and attitudes regarding the 
2-dose HPV vaccination schedule. 


METHODS 


From July 2018 to September 2018, 
we administered surveys to national 
networks of physicians who had 
agreed to participate in surveys about 
vaccine policy issues. The Colorado 
Multi-institutional Review Board 
approved this study as exempt 
research. 


Study Setting and Population 


As part of the Vaccine Policy 
Collaborative Initiative, a rapid 
survey mechanism used to assess 
physician attitudes about vaccine 
issues, we surveyed members of 
networks of pediatricians and FPs 
recruited from the memberships of 
the American Academy of Pediatrics 
(AAP) and the American Academy of 
Family Physicians (AAFP). Physicians 
were eligible if they spent =50% of 


their time providing primary care. We 
performed quota sampling“* to 
ensure networks were similar to the 
AAP and AAFP memberships with 
respect to region, urban versus rural 
location, and practice setting. In 
previous work, we demonstrated that 
survey responses from network 
physicians were similar to those of 
physicians randomly sampled from 
American Medical Association 
physician databases with respect to 
reported demographic characteristics, 
practice attributes, and attitudes 
about vaccination issues.” 


Survey Design 


We used 4-point Likert scales to 
assess physicians’ strength of 
recommendation of the HPV vaccine 
for patients in different age groups, 
physicians’ frequency of using 
different vaccine discussion styles, 
physicians’ perceived barriers to 
vaccination and extent of agreement 
with statements about the change 
from a 3-dose to a 2-dose schedule in 
younger teenagers, and physicians’ 
experiences with HPV vaccine 
delivery. Physicians were classified as 
using a “presumptive”*~ (or 
“announcement”)*” style if they 
reported almost always or always 
introducing the HPV vaccine by 
saying, “We've got 3 vaccines today: 
Tdap, HPV and meningococcal 
vaccines.’ Physicians were classified 
as using a “conversational”’"** (or 
“participatory”)*" style if they 
reported almost always or always 
saying, “Are you interested in getting 
HPV vaccine for your child today?” 
National advisory panels of 6 AAP 
members and 6 AAFP members 
representing different states 
pretested the survey, and it was then 
pilot tested among 13 pediatricians 
and 10 FPs. 


Survey Administration 


The survey was administered July 
2018 to September 2018, through the 
Internet or US mail, depending on 
physician preference. We sent the 
Internet group an initial e-mail with 
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up to 8 e-mail reminders, and we sent 
the mail group an initial mailing and 
up to 2 additional mail reminders. 
Nonrespondents in the Internet 
group were also sent up to 2 mail 
surveys in case of problems with 
e-mail. We patterned the mail 
protocol on Dillman’s Tailored Design 
Method.** 


Statistical Analysis 


We pooled Internet and mail surveys 
for analyses because studies have 
revealed that physician attitudes are 
similar when obtained by different 
methods.**~** Physicians who did not 
deliver the HPV vaccine in their 
practice were excluded from analysis. 
We compared respondents with 
nonrespondents using t tests for 
continuous variables and Pearson’s x” 
tests for categorical variables. A 
multivariable analysis was conducted, 
with the outcome of reporting 

a =50% refusal or deferral of HPV 
vaccination rate among 11- to 12- 
year-old patients within each sex. We 
used a log-Poisson model with robust 
error estimation to calculate relative 
risks. Independent variables included 
physician and practice characteristics 
(specialty, number of providers in 
practice, proportion of adolescent 
patients, proportion of privately 
insured patients) and physician HPV 
vaccine recommendation style and 
delivery experiences. We used 

a cutoff of P < .25 for inclusion of 
variables into the model. We used 

a stepwise backward elimination 
procedure, in which the least 
significant predictor in the model was 
eliminated sequentially. At each step, 
estimates were checked to make sure 
other variables were not affected by 
dropping the least significant 
variable. This resulted in the 
retention of only those factors that 
were significant at P < .05 in the final 
model, with the exception of specialty, 
which we forced into the final model. 
All analyses were performed by using 
SAS version 9.4 (SAS Institute, Inc, 
Cary, NC). 


RESULTS 


The overall response rate was 65% 
(588 of 908); 70% (317 of 456) 
among pediatricians and 60% (271 of 
452) among FPs. Of respondents, 280 
(48%) responded via the Internet. 
Fifteen pediatricians and 43 FPs did 
not administer the HPV vaccine and 
were excluded. Reasons for not 
administering the HPV vaccine were 
not explored in this survey. In 

Table 1, we compare respondents and 
nonrespondents and show additional 
characteristics available only for 
respondents. Among FPs, 
respondents were significantly 
younger than nonrespondents and 
had more providers in their practices. 


Delivery and Communication 
Practices for the HPV Vaccine 


Within each specialty, a larger 
percentage of physicians made 

a strong recommendation for HPV 
vaccination for older adolescents 
compared with adolescents 11 to 

12 years old (Fig 1). For each age 
group, a larger percentage of 
pediatricians than FPs made a strong 
recommendation. Sixty-five percent of 
pediatricians and 42% of FPs 
reported almost always or always 
using a presumptive style, whereas 
16% of pediatricians and 24% of FPs 
almost always or always used 

a conversational style of introduction. 
Overall, 40% of physicians used 
standing orders for HPV vaccination, 
66% had a computer-based system 
that could report adolescents who 
needed an HPV vaccine, and 42% had 
an electronic alert in the medical 
record if a patient needed an HPV 
vaccine (no significant differences by 
specialty). 


Attitudes and Experiences With HPV 
Vaccine Delivery 


A majority in both specialties strongly 
agreed or somewhat agreed that it is 
important that adolescents be 
vaccinated before they engage in 
physical intimacy (including kissing), 
that they encounter more resistance 


to HPV vaccination compared to other 
adolescent vaccines because it is not 
required by schools in their state, and 
that they encounter less resistance 
from patients and parents to 
beginning the HPV vaccine series at 
age 13 years than at age 11 years 
(Fig 2). Approximately one-third of 
physicians in both specialties 
reported anticipating an 
uncomfortable conversation if they 
recommended vaccination for 
patients at age 11 to 12 years. 


Rates of Refusal or Deferral of the 
HPV Vaccine 


Physicians reported high rates of 
refusal or deferral among 11- to 12- 
year-old patients, with lower levels 
among older adolescents (Fig 3); FPs 
reported higher rates than 
pediatricians at all patient ages, and 
physicians in both specialties were 
significantly more likely to report 
refusal or deferral for their male 
patients than for their female 
patients. 


Factors significantly associated with 
physicians reporting a =50% refusal 
or deferral rate among 11- to 12- 
year-old patients included the 
following: not strongly 
recommending to 11- to 12-year-old 
patients, not almost always or always 
using a presumptive recommendation 
Style, strongly agreeing that they 
encounter less resistance to HPV 
vaccination from patients age 

13 years versus patients age 11 years, 
and anticipating an uncomfortable 
discussion when recommending to 
11- to 12-year-old patients (Table 2). 
The multivariable results for both 
sexes were similar. 


Perceived Barriers to HPV Vaccine 
Delivery 


Perceived barriers to HPV vaccination 
that =20% of physicians in either 
specialty reported as major were the 
following: misinformation parents 
receive from the Internet or social 
media, parental concerns about the 
safety of the HPV vaccine, parents not 
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TABLE 1 Respondent and Nonrespondent Characteristics by Physician Specialty 


Characteristic 


Mean (SD)/median age, y 

Mean (SD)/median No. 
providers 

Male sex, % 

Region, % 

Midwest 
Northeast 
South 
West 

Location of practice, % 
Rural 
Urban (not inner city) 
Urban (inner city) 

Setting, % 

Private practice 
University, hospital, 

public, or other 
HMO 

Decision-making 
Independent 
Larger system level 

Proportion of adolescents 
11-18 y old in practice, % 
0%-9% 

10%—-19% 
20%—29% 
= 30% 

Proportion of Hispanic 
patients in practice, % 
0%—-9% 

10%—-24% 
20%—49% 
=50% 

Proportion of African 
American patients in 
practice, % 

0%—-9% 
10%—-24% 
20%—49% 
=50% 

Proportion of patients with 
private insurance in 
practice, % 

0%—24% 
20%—49% 
90%—74% 
75%—100% 

Proportion of patients with 
Medicaid or CHIP in 
practice, % 

0%—-9% 
10%—-24% 
20%—49% 
=50% 


Pediatricians 


Respondents 
(n = 317) 


02 (10)/52 
10 (16)/6 


56 


21 
25 


őő 
18 


Nonrespondents 
(n = 139) 


o1 (11)/50 
16 (55)/6 


54 


22 
19 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


Respondents 
(n = 271) 


06 (8)/59* 
12 (39)/6** 


og 


ól 
16 


69 
20 


41 
24 
19 
16 


FPs 


Nonrespondents 
(n = 181) 


08 (8)/58* 
7 (10)/5** 


60 


27 
16 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


N/A 
N/A 
N/A 
N/A 


CHIP, Children’s Health Insurance Program; HMO, health maintenance organization; N/A, not applicable. 
“P< 05 for t test for difference within specialty for respondents versus nonrespondents. 
“P< 05 for Wilcoxon rank test for difference within specialty for respondents versus nonrespondents. 


thinking the HPV vaccine was 
necessary for their daughters or sons, 
and opposition to vaccination for 
moral or religious reasons (Fig 4). 


Knowledge, Practices, and Attitudes 
Regarding 2-Dose Schedules 


Physicians, especially FPs, frequently 
were incorrect or reported not 
knowing about the number of doses 
recommended in different scenarios 
or whether additional vaccination 
with 9 valent HPV should be offered 
to adolescents who were fully 
immunized previously with 2 valent 
HPV or 4 valent HPV (Fig 5). 
Regarding dosing intervals, among 
pediatricians, 74% reported routinely 
recommending the second HPV dose 
6 months after the first, and 25% 
reported routinely recommending it 
at 12 months; among FPs, 
corresponding percentages were 88% 
at 6 months and 12% at a 12 months 
(P < .0001). 


The vast majority of physicians 
strongly or somewhat agreed that the 
2-dose schedule is as efficacious as 

a 3-dose schedule for adolescents 
<15 years and that the new schedule 
facilitated completion and initiation 
of the series among adolescents in 
this age group, with pediatricians 
being more likely to strongly agree 
with all of these statements than FPs 
(Fig 6). However, more than half of 
FPs and close to one-third of 
pediatricians strongly or somewhat 
agreed that having different 
recommendations for adolescents 
=15 years old and younger 
adolescents created confusion among 
patients and parents or practice staff. 


DISCUSSION 


Despite the existence of highly 
effective and safe HPV vaccines, 
national vaccination coverage 
remains suboptimal. This is the first 
national survey of physician attitudes 
regarding the new ACIP 
recommendations for a 2-dose 
schedule in younger adolescents, and 
it offers a glimpse into physician 
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FIGURE 1 


0% 20% 40% 


60% 80% 100% 


Strength of physician recommendation for HPV vaccine by patient age and sex (pediatricians, n = 502; FPs, n = 228). All comparisons between male and 
female patients within each age group within specialty differ significantly at P < .05 (Fisher’s exact test). * P < .05 for comparison between FPs and 


pediatricians (Fisher’s exact test). 


practices with respect to current HPV 
vaccine recommendations and 
delivery practices. Our data reveal 
that a high proportion of physicians 
are recommending the HPV vaccine, 
although more are recommending it 
for older adolescents than for those 
11 to 12 years old. A lower 
proportion of FPs compared with 
pediatricians strongly recommend the 
HPV vaccine to patients of all ages 
evaluated. Slightly more than half of 
pediatricians and less than half of FPs 
reported using a presumptive 
recommendation style, and less than 
half of those in both specialties used 
standing orders or electronic alerts in 
HPV vaccine delivery. Physicians 
reported high rates of refusal, 
especially by 11- to 12-year-old 
patients, who are the target 
population for vaccination according 
to national guidelines. Physicians 
were more likely to report high 
refusal rates by 11- to 12-year-old 
patients if physicians did not use 

a presumptive style, if physicians did 
not strongly recommend vaccination 
to young adolescents, or if physicians 
anticipated less resistance to 
vaccination from patients older than 


11 or 12 years. Although physicians 
demonstrated some knowledge gaps 
about the 2-dose recommendations, 
the majority believed that the revised 
schedule was facilitating both 
initiation and completion of the HPV 
vaccine series among adolescents 
<15 years of age. 


A physician recommendation has 
been shown to be 1 of the most 
important factors in parental vaccine 
acceptance,” ?? and the lack of 

a strong recommendation has been 
identified as an important barrier to 
HPV vaccination.?**°-*° Our data 
reveal substantial increases in the 
strength of recommendation when 
compared with a survey conducted 
~5 years ago by using similar 
methodology.*” Percentages of 
pediatricians strongly recommending 
the HPV vaccine have increased from 
60% in the 2013 survey to 85% in 
the 2018 survey for 11- to 12-year- 
old girls and from 52% to 83% for 
11- to 12-year-old boys. Smaller 
increases have occurred among FPs 
during the same period (from 59% to 
72% for 11- to 12-year-old girls and 
from 41% to 66% for 11- to 12-year- 


old boys), although the disparity in 
strength of recommendations 
between specialties persists.*” 
Although the overall percentage of 
physicians who strongly 
recommended the HPV vaccine for 
patients age 11 to 12 years was high 
in the current study, it was 13 to 15 
absolute percentage points lower 
than that for patients age 13 to 

14 years old in both specialties. 


Lower rates of strong 
recommendations for HPV 
vaccination among FPs have 
important implications given their 
prominent role in adolescent health 
care delivery. A recent national 
analysis revealed that the proportion 
of adolescents seen by FPs increases 
from 25% among 11-year-old male 
patients to 57% among 18-year-old 
male patients and from 31% among 
11-year-old female patients to 41% 
among 18-year-old female patients.” 
The differences we observed by 
specialty are reflective of previous 
literature regarding childhood°*°° 
and adolescent*””’ °* vaccines and 
highlight the fact that FPs need to 
remain a major focus of vaccine 


0 
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It is important that 
adolescents be vaccinated 
before they engage in 
early physical intimacy, 
including kissing. 


| encounter more 

resistance to the HPV vaccine 
compared with other 
adolescent vaccines 

because it is not required 

for schools in my state.* 


| encounter less resistance 
from parents and patients to 
beginning the HPV series 

at age 13 years versus at 
age 11 years. 


| recommend the HPV vaccine 
more often for adolescents 

at higher risk for getting 

HPV.” 


When | think about 
recommending the HPV 
vaccine for 11- to 12-year-old 
patients, | anticipate having an 
uncomfortable 

conversation. 


| do not push hard for 
adolescents to be 
vaccinated with the HPV 
vaccine if they are not 
engaging in risky sexual 
behaviors.” 


FIGURE 2 


Physician experiences with HPV vaccine discussions (pediatricians: n = 302; FPs: n = 228). * P < .05 for comparison between FPs and pediatricians 


Pediatrician 


Pediatrician 


Pediatrician 


Pediatrician 


Pediatrician 


Pediatrician i» GY 


E Strongly agree 


O Somewhat disagree 


E Somewhat agree 
O Strongly disagree 


FP 


FP 


27% 


FP 28% 


FP 


FP 
6% 


FP 


0% 20% 40% 


test). Some percentage do not add up to 100% because of rounding. 


education and practice improvement 
strategies to improve HPV vaccine 
delivery. 


Reported rates of deferrals or refusals 
were high among our respondents, 
especially for patients ages 11 to 

12 years and among FPs and appear 
relatively stable from 5 years ago.”” 
Physicians, especially FPs, reported 
higher rates of deferral or refusal for 
their male patients compared with 


female patients in every age group. 
This may be related to the longer 
duration of recommendations for 
female patients or to a higher 
perceived benefit of the HPV vaccine 
for female patients. Physicians who 
perceive higher refusal or deferral 
rates among 11- to 12-year-old 
patients more frequently were those 
who (1) did not strongly recommend 
vaccination in this age group, (2) 
anticipated uncomfortable 


~j 
oS 


8% 


9% 


6% 


54% 


39% 


17% 


62% 


60% 80% 100% 


2 
X 


~ 


discussions with parents of 11- to 12- 
year-old patients, and (3) expected to 
encounter more resistance from 
parents of 11- to 12-year-old patients 
versus older adolescents. This finding 
is consistent with those of previous 
studies demonstrating that physicians 
are less likely to discuss the HPV 
vaccine with patients aged 11 to 

12 years if they believe parents are 
likely to defer,*?°*°* However, other 
literature has revealed that 
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FIGURE 3 


Male sex* 


m>50% 


0% 20% 40% 


025% to 49% 
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Proportion of physicians reporting patients and/or parents refusal or deferral of the HPV vaccine, by patient age and sex (pediatricians: n = 302; FPs: 
n = 228). All comparisons between male and female patients within each age group within specialty differ significantly at P < .05 (x? test). Some bars 
do not add up to 100% because of rounding. * P < .05 for comparison between FPs and pediatricians (x? test). 


physicians may considerably 
overestimate the amount of 
resistance to vaccination that they are 
likely to encounter.” Our data also 
reveal that physicians who report 
high levels of refusal or deferral are 
also less likely to use a presumptive 
recommendation style. To our 
knowledge, this has not previously 
been reported. The circular nature of 
provider anticipation of refusal or 
deferral potentially leading to 

a weaker recommendation style and 
less persistence in responding to 
parental hesitancy®® could be 
creating a self-perpetuating cycle 
within a subgroup of physicians. 


Our data suggest that improvements 
are needed in how the HPV vaccine is 
being recommended. A 
presumptive’ style of initiating HPV 
vaccine discussions uses words that 
convey an assumption of vaccination 
and does not discuss the HPV vaccine 
in a different manner than other 


adolescent vaccines, whereas, 

a conversational®” style engages 
parents in an open-ended discussion 
about the HPV vaccine without 
linguistic presupposition of 
vaccination. A presumptive approach 
has been shown to be associated with 
higher HPV acceptance compared 
with a conversational approach in 
multiple studies.*’-** Updated 
implementation guidance from the 
AAP includes the use of a strong 
pediatrician recommendation with 
wording consistent with 

a presumptive style,’ and the AAFP 
Web site refers FPs to Centers for 
Disease Control and Prevention 
talking points that included 
recommendations for “same way” as 
for other adolescent vaccines and 
“same day’ recommendations. Our 
data indicate that large percentages 
of physicians, including a majority of 
pediatricians, have adopted the 
presumptive communication style in 
initiating HPV vaccine discussion, but 


there are still substantial proportions 
who have not. In addition to 
improving physician communication 
styles, HPV vaccine delivery could 
also be optimized by increased use of 
evidence-based methods, including 
standing orders and alert systems in 
the medical record to remind 
providers of the need for vaccination 
at the point of care, both of which are 
recommended by the Community 
Preventive Services Task Force.°”°® 


Factors reported by physicians as 
important barriers to HPV 
vaccination (including perceived 
parental concerns about safety or 
effects of vaccination on sexual 
behavior and parents thinking that 
the HPV vaccine is not necessary or 
that there are too many vaccines at 1 
visit) are consistent with previously 
identified barriers.** However, the 
barrier most frequently reported by 
physicians in both specialties (the 
effect of misinformation parents 
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FIGURE 4 

Physicians perceived barriers to HPV vaccination (pediatricians: n = 502; FPs: n = 228). Some bars do not add up to 100% because of rounding. Barriers 
reported as major by <12% by both specialties include the following: parent concerns that vaccination may encourage their sons to have earlier sexual 
behavior, that vaccination may encourage their sons to have riskier sexual behavior, that the vaccine could cause infertility in their daughters and/or 
sons, or that their child will suffer immediate short-term effects from the HPV vaccine; parent concerns about the efficacy of the HPV vaccine and about 
waning immunity if the HPV vaccine is given too early; physician concern about giving too many vaccines in 1 visit (it will result in an uncomfortable 
conversation with the parent), about the safety of the HPV vaccine for female patients, and about the efficacy of the HPV vaccine for male and female 
patients; physician belief that HPV infection is not common enough in male and female patients to justify a vaccination, that Papanicolaou tests are an 
adequate way to prevent cervical cancer, and that HPV-associated diseases are not severe enough in male and female patients to justify a vaccination; 
parents wanting to wait to begin the HPV series until after menarche for girls; failure of some insurance companies to cover HPV vaccination; the up-front 
costs for a practice to purchase the vaccine; lack of adequate reimbursement for vaccination; and the time it will take a physician to discuss HPV 
vaccination with his or her patients and their parents. * P < .05 for comparison between FPs and pediatricians (x? test). 


69-72 


receive from the Internet or social content, its contribution to low harms were more commonly found in 
media) is one that has not been HPV vaccination acceptance has not social media than in traditional media 
frequently highlighted. Although been well studied. A recent survey of or conversations and were associated 
social media have been recognized as 1263 parents of US adolescents with lack of initiation, delay, or 

a significant source of vaccine-critical revealed that stories of HPV vaccine refusal of HPV vaccination.’* Along 
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TABLE 2 Characteristics of Physicians Reporting a 50% or Higher Refusal or Deferral Rate for HPV Vaccination Among Their 11- to 12-Year-Old Patients by 


Sex 
Female Patients (n = 482) Male Patients (n = 497) 
<50% Refusal =90% Refusal MV RR <50% Refusal =950% Refusal MV RR 
or Deferral Rate or Deferral Rate (95% Cl) or Deferral Rate or Deferral Rate (95% Cl) 
(n = 376; 78%) (n= 106; 22%) (n = 356; 72%) (n= 141; 28%) 
Physician and practice characteristics 
Practice specialty 
FP 39% 50% 1.11 58% 04% 1.30 
(0.81—1.52) (0.99—1.69) 
Pediatrician 61% 00% Reference 62% 46% Reference 
No. providers in practice, median (IQR) 6 (4-11) 5 (3-9) N/A 6 (4-11) 5 (3-10) N/A 
Proportion of adolescents 11—18 y old in practice, % 
0%—-9% 20 31 i 21 29 j 
=10% 80 69 N/A 79 T1 N/A 
Proportion of patients with private insurance in 
practice, % 
0%—24% 26 16 a 20 22 i 
=25% 74 84 N/A T5 78 N/A 
HPV recommendation style 
Strongly recommend to adolescents 11—12 y old 
Yes 88% 01% Reference 86% 02% Reference 
No 12% 49% 2.14 14% 48% 1.90 
(1.50—3.04) (1.43-2.53) 
Use of a presumptive recommendation style 
Frequently, occasionally, rarely, or never 56% 66% 1.61 54% 67% 1.74 
(1.11-2.32) (1.27—2.38) 
Almost always or always 64% 54% Reference 66% 55% Reference 
Delivery experiences and attitudes 
Perceive less resistance from parents and patients 
to beginning the HPV series at age 15 y versus at 
age 11 y 
Strongly agree 20% 04% 2.55 20% 47% 1.84 
(1.69-—3.22) (1.42—2.38) 
Other 80% 46% Reference 80% 05% Reference 
Do not push hard for adolescents to be vaccinated with i 2 
HPV vaccine if they are not engaging in risky sexual 
behaviors, % 
Strongly or somewhat agree 8 25 N/A 8 20 N/A 
Strongly or somewhat disagree 92 TT N/A 92 80 N/A 
Anticipate having an uncomfortable conversation when 
recommending HPV vaccine for adolescents 11—12 y 
old 
Strongly or somewhat agree 50% 01% 1.62 50% 46% 1.52 
(1.17-2.24) (1.02—1.72) 
Strongly or somewhat disagree 70% 49% Reference 70% 04% Reference 


Cl, confidence interval; IQR, interquartile range; MV, multivariable regression; RR, relative risk; N/A, not applicable. 
a Variables were not significant in the multivariable model and were therefore excluded from the final model. 


with these data, our findings highlight it is not surprising that there were presumably, the ability of the 

the importance of developing knowledge gaps for proportions of electronic medical record or 
effective public health physicians about when a third dose is immunization registry—based systems 
communication strategies to recommended, the acceptability of to forecast the need for HPV 
counter misinformation and highlight using different HPV vaccines to vaccination. Interestingly, the 
benefits of HPV vaccination via complete the series, and the need for majority of physicians in both 
social media. an additional 9 valent HPV specialties reported recommending 
Given the relative recency of vaccination in those already fully the second dose of the 2-dose 
recommendations for the 2-dose vaccinated with either 2 valent HPV schedule at a 6-month interval 
schedule for adolescents initiating or 4 valent HPV. Physician knowledge rather than 12 months later at 
HPV vaccination before age 15 years, should increase over time, as will, the next annual well-child visit, 
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67% 
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Pediatrician FP 


Adolescents of any age previously 
vaccinated with 2 doses of 4vHPV 2 
months apart should receive a third 
dose of 9VHPV* (true). 


O Incorrect 


& Do not know or not sure 


69% 


48% 
35% 
15% 16% [E 
FP 


Pediatrician 


If adolescents began the HPV 
series with the 2vVHPV or 4vHPV 
vaccines and need additional doses, 
they need to complete the entire 
9vHPV series* (false). 


Physician knowledge regarding 2-dose HPV vaccine schedules (pediatricians: n = 302; FPs: n = 228). * P < .05 for comparison between FPs and 
pediatricians (x? test). 2vHPV, 2 valent HPV; 4vHPV, 4 valent HPV; 9VHPV, 9 valent HPV. 


although national guidelines allow 
for either. This may reflect an 
understanding of the importance 
of series completion as soon as 
possible. Importantly, primary 
care physicians strongly endorsed 
the belief that the 2-dose 
schedule was facilitating both 
initiation and completion of the 
HPV series. 


There are important strengths and 
limitations to our data. We surveyed 
large, nationally representative 
samples of pediatricians and FPs and 
achieved high response rates. 
Responses of our sentinel physicians 
may not be fully generalizable, 
although previous work has 
revealed that the survey methods 
described yield similar responses to 
the most commonly employed 
method of sampling physicians 
nationally.2* Nonrespondents might 
have different views than 
respondents, although our high 


response rates helps to mitigate 
against this source of bias. Most 
importantly, our data are based 

on self-report rather than 

direct observation and may not 

entirely reflect actual physician 

practice. 


This snapshot of HPV vaccine delivery 
in primary care demonstrates room 
for improvement in the way 
physicians are communicating about 
the HPV vaccine and in their delivery 
practices. Most important in this 
regard is the continued finding that 
physicians who experience or expect 
to experience high rates of deferral or 
refusal may be anticipating and 
accommodating refusals by altering 
their recommendation strength and 
style. Such accommodations by 
physicians may perpetuate a lack of 
acceptance of the HPV vaccine among 
parents. Greater physician awareness 
about the potential of overestimating 
the degree of parental resistance to 


HPV vaccination and about the 
effectiveness of a strong 
recommendation for the HPV vaccine, 
delivered in the same way as for 
other adolescent vaccines and on 
same day as other adolescent 
vaccines, may be key to increasing 
acceptance among parents of 11- to 
12-year-old patients. Increased use of 
available communication training 
materials and applications as well as 
further development of evidence- 
based messages for parents may 

be helpful in improving the way 
HPV vaccination is introduced.’*~”® 
Our data are encouraging in 
revealing substantial increases 

over the past 5 years in the 
percentage of physicians who 

report strongly recommending the 
HPV vaccine to 11- to 12-year-old 
patients. The findings also reveal 
that according to primary care 
physicians, the 2-dose schedule 
could result in meaningful 
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FIGURE 6 
Primary care physicians’ attitudes regarding the 2-dose HPV vaccine (pediatricians: n = 302; FPs: n = 228). Some bars do not add up to 100% because of 
rounding. ° Fisher’s exact test.* P < .05 for comparison between FPs and pediatricians (x? test). SVHPV, 9 valent HPV. 
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Parenteral Antibiotic Therapy Duration 
in Young Infants With Bacteremic 
Urinary Tract Infections 


Sanyukta Desai, MD,* Paul L. Aronson, MD, MHS,"°* Veronika Shabanova, PhD,®® Mark I. Neuman, MD, MPH 

Frances Balamuth, MD, PhD,®" Christopher M. Pruitt, MD, Adrienne G. DePorre, MD; Lise E. Nigrovic, MD, MPH,' 

Sahar N. Rooholamini, MD, MPH,“ Marie E. Wang, MD, MPH,' Richard D. Marble, MD," Derek J. Williams, MD, MPH," 
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YOUNG INFANT RESEARCH COLLABORATIVE 


OBJECTIVES: To determine the association between parenteral antibiotic duration and outcomes 
in infants <60 days old with bacteremic urinary tract infection (UTI). 


METHODS: This multicenter retrospective cohort study included infants <60 days old who had 
concomitant growth of a pathogen in blood and urine cultures at 11 children’s hospitals 
between 2011 and 2016. Short-course parenteral antibiotic duration was defined as <7 days, 
and long-course parenteral antibiotic duration was defined as >7 days. Propensity scores, 
calculated using patient characteristics, were used to determine the likelihood of receiving 
long-course parenteral antibiotics. We conducted inverse probability weighting to achieve 
covariate balance and applied marginal structural models to the weighted population to 
examine the association between parenteral antibiotic duration and outcomes (30-day UTI 
recurrence, 30-day all-cause reutilization, and length of stay). 


RESULTS: Among 115 infants with bacteremic UTI, 58 (50%) infants received short-course 
parenteral antibiotics. Infants who received long-course parenteral antibiotics were more 
likely to be ill appearing and have growth of a non-Escherichia coli organism. There was no 
difference in adjusted 30-day UTI recurrence between the long- and short-course groups 
(adjusted risk difference: 3%; 95% confidence interval: —5.8 to 12.7) or 30-day all-cause 
reutilization (risk difference: 3%; 95% confidence interval: —14.5 to 20.6). 


CONCLUSIONS: Young infants with bacteremic UTI who received <7 days of parenteral antibiotics 
did not have more frequent recurrent UTIs or hospital reutilization compared with infants 
who received long-course therapy. Short-course parenteral therapy with early conversion to 
oral antibiotics may be considered in this population. 


Oo 


WHAT’S KNOWN ON THIS SUBJECT: Infants =60 days old with 
bacteremic urinary tract infection often receive prolonged courses of 
parenteral antibiotics. The safety of short-course parenteral antibiotic 
therapy has not been established in this population. 
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Urinary tract infection (UTI) is 1 of 
the most common bacterial infections 
in infants <60 days of age.’~* 
Approximately 8% to 10% of young 
infants with UTI have concomitant 
bacteremia.”*” However, no 
evidence-based guidelines exist to 
direct management in this population, 
particularly regarding optimal 
parenteral (ie, intravenous and/or 
intramuscular [IV/IM]) antibiotic 
duration. 


Although several studies have 
supported the safety of short-course 
parenteral antibiotic therapy with 
early conversion to oral antibiotics in 
young infants with uncomplicated 
UTI,°°® the safety of short-course 
parenteral antibiotic therapy in young 
infants with bacteremic UTI has not 
been established. As a result, infants 
with bacteremic UTI often receive 
prolonged courses of parenteral 
antibiotics, which can lead to long 
hospitalizations and increased costs. 
Furthermore, the peripherally 
inserted central catheters (PICCs) 
often required to deliver prolonged 
parenteral antibiotic therapy expose 
infants to added risks, including 
catheter-associated bloodstream 
infections as well as mechanical and 
thrombotic central-line 
complications.’ The benefit of 
prolonged parenteral antibiotic 
therapy is unclear in this 
population.°*** 


Our objective for this study was to 
examine the association between 
parenteral antibiotic duration and 
clinical outcomes in infants <60 days 
old with bacteremic UTI. 


METHODS 


Study Design 


This study was nested within 

a multicenter retrospective cohort 
study’~** of young infants evaluated 
for invasive bacterial infections in the 
emergency departments (EDs) of 11 
geographically diverse children’s 
hospitals (Supplemental Table 3). The 


study was approved by each site’s 
institutional review board with 

a waiver of informed consent and 
permission for data sharing. 


Study Population 


Infants <60 days old who presented 
to the EDs of the 11 participating 
hospitals between July 1, 2011, and 
June 30, 2016, were included in the 
parent study if they had a bacterial 
pathogen isolated in blood or 
cerebrospinal fluid (CSF) culture. 
Pathogenic bacteria were determined 
a priori via literature review and 
expert consensus (Supplemental 
Table 4).'*?° For this substudy, we 
included infants from the parent 
study who were diagnosed with a UTI 
and had detection of the same 
bacterial pathogen in both blood and 
urine cultures (Fig 1). UTI was 
defined as growth of =50 000 colony- 
forming units (CFUs)/mL of 

a pathogen in urine culture or growth 
of =10 000 CFUs/mL of a pathogen in 
urine culture plus an abnormal 
urinalysis (defined as presence of 
leukocyte esterase, nitrites, or >5 
white blood cells per high-power field 
on urine microscopy).”"”"? We 
excluded infants who had a bag or 
unknown collection method for urine 
culture if pathogen growth was 
<100 000 CFUs/mL or if multiple 
organisms were isolated because 
these organisms could represent 
contaminants.’°*” Infants who had 
concomitant bacterial meningitis, 
infants with CSF pleocytosis who 
received antibiotic pretreatment, and 
infants with examination and 
laboratory findings consistent with 
focal infections, such as septic 
arthritis, were also excluded from our 
cohort because these infants would 
likely be prescribed prolonged 
courses of parenteral antibiotics for 
treatment of other invasive infections. 
Finally, we excluded infants who died 
and those who were transferred out 
from the participating institution 
before completion of parenteral 
antibiotic therapy because we could 


not report total duration of IV/IM 
antibiotic therapy in these patients. 


Data Sources 


Each site’s electronic medical record 
system or microbiology laboratory 
was queried to identify all infants 
who had a positive blood or CSF 
culture result collected in the ED 
during the study period. A medical 
record review was then conducted to 
confirm eligibility. Information on 
patient demographics, clinical 
variables (eg, presence of fever and 
physical examination findings), 
laboratory data, parenteral antibiotic 
duration, adverse events, and 30-day 
ED revisit or readmission were also 
obtained via detailed medical record 
review. 


Exposure 


The exposure of interest was duration 
of parenteral antibiotic therapy. 
Patients were divided into short- 
course and long-course exposure 
groups. Given that we found 

a somewhat bimodal distribution of 
parenteral antibiotic duration with 

a median duration of 7 days (Supple 
mental Fig 2), we defined short 
course as receipt of parenteral 
antibiotics for <7 days and long 
course as receipt of parenteral 
antibiotics for >7 days. 


Outcomes 


The primary outcome was recurrence 
of UTI that resulted in an ED revisit or 
hospitalization within 30 days of 
discharge from the index hospital 
admission. For infants who had 
recurrent UTI, we conducted 
additional medical record review to 
obtain information on timing of 
recurrence, cauSative organism at 
recurrence, presence of bacteremia or 
meningitis at recurrence, diagnosis of 
vesicoureteral reflux (VUR), and use 
of prophylactic antibiotics before 
recurrence. Secondary outcomes 
included hospital length of stay (LOS), 
30-day all-cause reutilization (defined 
as hospital readmission or ED revisit 
within 30 days of index discharge), 
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Infants <60 days of age with invasive 


bacterial infections 
N=497 


Infants with bacteremia 
n=417 


Bacteremic UTI cohort 
n=115 


n=58 n=57 


Infants who received 


<7 days of IV/IM 
antibiotics 


Infants with bacteremia and UTI 
n= 140 
Infants with bacteremic UTI 
n= 131 


No bacteremia 
n=90 


Different organisms in blood 
and urine cultures 
n=9 


Excluded: n = 16 
Meningitis: n = 6 
CSF pleocytosis with 
pretreatment: n = 4 
Bag urine specimen”: n = 3 
Septic arthritis: n = 1 
Death: n = 1 
Transferred before antibiotic 


completion: n = 1 


Infants who received 
>7 days of IV/IM 
antibiotics 


FIGURE 1 


Flowchart of the study population. ° UTI definition: positive urine culture result with =50 000 CFUs/ 
mL or 210000 CFUs/mL with an abnormal urinalysis (leukocyte esterase-positive or nitrite-positive 
or >5 WBCs). ° Patients who had a bag or unknown method of urine culture collection were 
excluded if the culture grew <100000 CFUs/mL or if >1 organism was isolated. WBC, white 


blood cell. 


and adverse events within 30 days of 
discharge (ie, readmission to an ICU, 
need for mechanical ventilation or 
vasopressor use, or neurologic 
sequelae). 


Covariates 


Covariates of interest included 
patient age, sex, prematurity (defined 
as gestational age <37 weeks), 
complex chronic condition, ® known 
genitourinary anomaly, presence of 
fever (ie, a reported temperature 
=38.0°C (100.4°F) at home, in an 


outpatient clinic, or documented in 
the ED and obtained via any method), 
ill appearance, ICU stay during the 
index admission, causative pathogen, 
and prolonged bacteremia. Ill 
appearance was defined as any of the 
following physical examination terms 
documented on arrival to the ED: “ill 
appearing,” “toxic,” “limp,” 


“unresponsive, “gray, “cyanotic,” 


“apnea, “weak cry,’ “poorly perfused,’ 
“grunting, “listless,” “lethargic,” or 
“irritable.”’? If none of these terms 


were documented, the infant was 


classified as not ill appearing. In cases 
of contradicting documentation of ill 
appearance between attending 
physician and trainee, the attending 
physician’s documentation was used. 
Prolonged bacteremia was defined as 
isolation of the initial pathogen on 

a repeat blood culture drawn at least 
24 hours after the initial culture.”° 


Statistical Analysis 


Categorical variables were described 
by using frequencies and percentages. 
To adjust for potential confounding 
by indication for duration of 
parenteral antibiotic therapy, we 
generated a propensity score for each 
patient to estimate the conditional 
probability of receiving long-course 
parenteral antibiotic therapy.’ t”? 
Propensity scores were calculated by 
using a multivariable logistic 
regression model that consisted of 
baseline demographic characteristics 
(ie, age and sex), past medical history 
(ie, prematurity, presence of chronic 
complex condition, and known 
genitourinary anomaly), and clinical 
characteristics at presentation (ie, 
fever, ill appearance, ICU stay, 
prolonged bacteremia, and causative 
organism). 


We then conducted inverse 
probability weighting (IPW), aiming 
to achieve balance in the distribution 
of covariates between exposure 
groups by multiplying each study 
observation by the inverse of its 
respective propensity score to 
achieve a Statistically balanced 
pseudo-study population.** 
Distributional balance of covariates 
between short-course and long- 
course antibiotic groups after the 
weighting procedure was examined 
by using standardized mean 
differences, with the magnitude of 
<0.1 indicating acceptable balance 
(Supplemental Table 5).2* One 
variable, prolonged bacteremia, 
remained unbalanced; however, it 
was neither significantly associated 
with the outcomes of recurrent UTI 
and reutilization, as tested by Fisher's 
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exact test, nor with LOS, as tested by 
the Wilcoxon rank test. To 
incorporate the effect of study site, 
we attempted 2 additional models to 
generate propensity scores: 1 in 
which study site was used as a fixed- 
effect predictor in the above logistic 
regression and a second in which 

a random intercept was added for 
each study site. However, neither 
model yielded adequate balance 
between covariates. Thus, the 
multivariable logistic regression 
model without study site 
(Supplemental Table 5) was used for 
the analysis. 


To evaluate the association of 
antibiotic duration with our outcomes 
of interest, we employed marginal 
structural models and applied them 
to our weighted population.** We 
used the logit link function to 
estimate the probability of recurrent 
UTI and reutilization in each group 
and summarized our findings using 
both relative risk and risk difference. 
We used the identity link to estimate 
the adjusted difference in the mean 
LOS between short-course and long- 
course treatment groups. Natural log 
transformation was applied to LOS 
before its modeling in the marginal 
structural models. Statistical 
significance was established at the 2- 
sided a level of .05. Data were 
analyzed by using R version 3.4.3 
statistical software** and by using the 
following statistical packages: lme4,7° 
tableone,~° ipw,*’ sandwich,*® and 
survey.” 


RESULTS 


Patient Characteristics 


Of the 497 infants <60 days old in 
the parent study cohort who had 

a pathogen isolated in blood or CSF 
culture, we identified 140 infants 
with bacteremia and UTI caused by 
the same pathogen. Applying 
exclusion criteria, we assembled 

a cohort of 115 infants <60 days of 
age with bacteremic UTI (Fig 1). Of 
this cohort, 52% of infants were 


<28 days of age, 60% were boys, and 
9% were premature (Table 1). A 
subset of patients had a complex 
chronic condition or known 
genitourinary anomaly on admission, 
and the majority of infants presented 
with fever (Table 1). 


Urine culture was obtained via 
catheterization for 110 (96%) infants 
in our cohort. Escherichia coli was the 
most common causative organism 
and was found in 93 infants (81%). 
Six infants (5%) grew Enterococcus 
faecalis, 5 (4%) grew Staphylococcus 
aureus, 4 (3%) grew Klebsiella spp, 3 
(3%) infants grew Streptococcus 
agalactiae, 3 (3%) grew Enterobacter 
spp, and 1 (1%) grew Citrobacter spp 
from blood and urine cultures. Of 
note, all 5 infants with S aureus 
bacteremia were eventually 
diagnosed with urinary tract 
anomalies. 


We included a total of 5 patients 
growing >100 000 CFUs/mL of E coli 
in a bagged urine specimen. Of these, 
4 had evidence of pyuria, and 1 
patient did not have a urinalysis 
conducted. All patients grew E coli in 
the blood with identical antibiotic 
susceptibilities. 


Eight infants (7%) had prolonged 
bacteremia; all 8 patients with 
prolonged bacteremia in our cohort 
grew organisms that were sensitive to 
the empirical antibiotics they had 


been receiving since admission. No 
patients grew extended-spectrum 
8-lactamase-producing organisms in 
a blood or urine culture. All 
pathogens identified in our cohort 
were susceptible to an antibiotic with 
an available oral alternative. 


The median duration of parenteral 
antibiotic therapy was 7 days (range: 
2-24 days). The 3 most common 
parenteral treatment durations were 
14 days (27 infants; 23%), 7 days (18 
infants; 16%), and 10 days (13 
infants; 11%) (Supplemental Fig 2). 
Overall, 58 (50%) infants received 
short-course parenteral antibiotics 
(<7 days), and 57 (50%) infants 
received long-course parenteral 
antibiotics. The proportion of infants 
receiving short-course parenteral 
antibiotic therapy varied by 
institution, with a median of 50% 
(range: 11%-81%). Infants receiving 
long-course parenteral antibiotic 
therapy were more likely to be ill 
appearing on presentation, have 
growth of a non-E coli organism, and 
have prolonged bacteremia (Table 1). 


Outcomes 


Six infants (5%) had a recurrent UTI, 
each diagnosed between 15 and 

30 days after index discharge. Two of 
the 6 infants received short-course 
parenteral therapy, and 4 received 
long-course parenteral antibiotic 
therapy during the index 


TABLE 1 Baseline Demographic and Clinical Characteristics of Infants With Bacteremic UTI 


Patient Characteristics Overall Short-Course IV/IM Long-Course IV/IM P (x? 

(N = 115) Antibiotics Antibiotics Test) 

(n = 58) (n = 57) 
n (%) n (%) n (%) 
Age =28 d 60 (52) 20 30 (61) 05 
Male sex 69 (60) 30 39 (68) 06 
Prematurity 10 (9) 3/53 (6) 7/54 (13) 38° 
Complex chronic 19 (17) 7 (12) 12 (21) 19 
condition 

Genitourinary anomaly 17 (15) 6 (10 11 (19) 18 
Fever 102 (89) ol 91 (89) 19 
Ill appearance 32 (28) 10 22 (39) 01 
ICU stay 16 (14) 5 11 (19) 10 
Prolonged bacteremia 8 (7) 8 (14) 002° 
Non—E coli organism 22 (19) 4 18 (32) 001° 


a Fisher’s exact test was performed. 
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hospitalization (Table 2). There was 
no significant difference in rates of 
recurrent UTI between the 2 
treatment groups (adjusted risk 
difference: 3%; 95% confidence 
interval [CI]: —5.8 to 12.7). Both 
infants in the short-course group who 
had a recurrent UTI also had 

a diagnosis of VUR (Supplemental 
Table 6). One infant in the short- 
course group had recurrent E faecalis 
bacteremia. This patient had grade 3 
to 4 VUR diagnosed during 
hospitalization for the recurrent UTI 
and had not been receiving 
prophylactic antibiotics after 
discharge from the index hospital 
admission (Supplemental Table 6). 
Two of the 4 infants in the long- 
course group with recurrent UTI had 
grade 4 VUR and were receiving 
prophylactic amoxicillin at the time of 
recurrence. None of the 6 infants with 
recurrent UTI developed meningitis. 


Two of the 6 infants with recurrent 
UTI grew an organism distinct from 
the causative organism isolated at the 
index hospitalization (Supplemental 
Table 6). Both of these infants had 
received long-course parenteral 
antibiotic therapy during their index 
hospital stay. When evaluating only 
those infants who had a recurrent UTI 
with growth of the same pathogen 
that caused their index UTI, there was 
no difference in rates of recurrence 


TABLE 2 Outcomes of Infants With Bacteremic UTI 


between the 2 treatment groups 
(adjusted risk difference: 0.2%; 95% 
CI: —7.8 to 8.3) (Table 2). 


Fifteen infants (13%) had 30-day all- 
cause reutilization. There was no 
significant difference in the rates of 
reutilization between the 2 treatment 
groups (adjusted risk difference: 3%; 
95% CI: —14.6 to 20.4). The adjusted 
mean LOS was significantly longer in 
the long-course group compared with 
the short-course group (adjusted 
mean difference: 6 days; 95% CI: 4.0 
to 8.8). 


Within our cohort, 67 infants were 
well appearing and had no known 
complex chronic condition, including 
genitourinary anomaly. Short-course 
parenteral antibiotic therapy was 
prescribed in 40 (60%) healthy, well- 
appearing infants. Two infants (5%) 
in the short-course group and 2 
infants in the long-course group (7%) 
had a recurrent UTI. 


No infants in our cohort experienced 
adverse events such as ICU 
readmission, need for mechanical 
ventilation or vasopressor use, or 
signs of neurologic sequelae within 
30 days of discharge from the index 
hospitalization. Thirteen infants in 
the long-course group were 
discharged from the hospital with 

a PICC to complete intravenous (IV) 
antibiotic therapy. Of these, 1 infant 


Outcome Short-Course IV/ _Long-Course IV/IM Adjusted Percent Adjusted 
IM Antibiotics Antibiotics Relative Adjusted Risk Mean 
(n = 58) (n = 57) Risk (95% Difference Difference 
cl)? (95% CI)? (95% CI)? 
Recurrent UTI, n 2 (5) 4 (7) 1.9 (0.5 to ő (—5.8 to — 
(%) 11.6) 12.7) 
Recurrent UTI 2 (3) 2 (4) 1.1 (0.1 to 0.2 (—7.8 to — 
with same 8.4) 8.3) 
organism, n 
(%) 
All-cause 6 (10) 9 (16) 1.2 (0.4 to 5 (—14.5 to — 
reutilization, n 5.9) 20.6) 
(%) 
LOS, d, adjusted 4.5 (4.4 to 4.6) 10.8 (10.7 to 10.9) — — 6 (4.0 to 8.8) 


mean (95% 
Cl) 


a IPW was conducted by using propensity scores to achieve covariate balance in patient clinical and demographic 
characteristics. Marginal structural models were applied to the weighted population to obtain adjusted estimates. 


(8%) had an ED revisit for a PICC- 
related mechanical complication. 


DISCUSSION 


In this multicenter study of young 
infants with bacteremic UTI, shorter 
courses of parenteral antibiotic 
therapy (ie, =7 days) were not 
associated with significantly more 
frequent UTI recurrence or all-cause 
reutilization when compared with 
longer courses of parenteral antibiotic 
therapy. Adverse events were rare in 
both groups, and longer courses of 
antibiotic therapy were associated 
with longer lengths of hospital stay. 


A previous multicenter retrospective 
study by Schroeder et al?’ of infants 
<90 days old with bacteremic UTI 
revealed that infants with recurrent 
UTI did not receive longer parenteral 
antibiotic treatment when compared 
with infants without recurrent UTI in 
an unadjusted analysis. Similar to our 
study, Schroeder et al? reported that 
infants who received longer courses of 
parenteral antibiotics were more likely 
to be younger, have growth of a non-E 
coli organism in blood and urine 
cultures, and have blood culture 
positivity for >1 day. Because infants 
who were perceived to have higher- 
risk baseline characteristics received 
longer courses of parenteral antibiotic 
therapy, we used propensity scores 
and IPW to mitigate potential 
confounding by indication. 


Our study had several limitations. 
First, despite the large number of 
centers included, we identified 
relatively few infants with bacteremia 
in the context of UTI. Within our 
cohort, rates of recurrent UTI and 
reutilization were low. Therefore, our 
analysis was not powered to capture 
small but potentially clinically 
important differences in outcomes. 
However, given that the incidence of 
30-day adverse outcomes for infants 
with short-course antibiotics in our 
study was low, if modest UTI 
recurrence differences did actually 
exist, they might prove tolerable for 
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a clinician weighing the 
aforementioned risks associated with 
prolonged parenteral antibiotic 
therapy. 


There was also potential for 
misclassification in infants with 
recurrent UTI because patients whose 
recurrence was managed in the 
outpatient setting or at a different 
institution would not have been 
captured by using our data collection 
methodology. This misclassification 
could lead to an underestimation of 
the rate of recurrence. However, given 
the history of previous bacteremia in 
our cohort, our patient population is 
less likely to be managed in the 
outpatient setting when compared 
with infants who have uncomplicated 
UTI. Additionally, the overall rate of 
recurrence of UTI in our study was 
similar to that of previous studies of 
infants with bacteremic UTI.°°°* 


Furthermore, we could not address 
optimal total duration of therapy for 
bacteremic UTI in this study because 
we did not have information on 
choice or duration of postdischarge 
oral antibiotic therapy for our cohort. 


Finally, it is possible that infants in 
the short-course parenteral antibiotic 
group received longer courses of oral 
antibiotics after discharge compared 
with infants in the long-course 
parenteral group. We were not able to 
evaluate for recurrences of UTI from 
completion of oral therapy using our 
data source. Thus, it is possible that 
a recurrent UTI that occurred 30 days 
after completion of oral antibiotic 
therapy in the short-course group 
was not captured in our 30-day 
postdischarge window. 


The safety of short-course parenteral 
antibiotic therapy with early 
conversion to oral antibiotics has 
been established for the treatment of 
many invasive pediatric bacterial 
infections (eg, complicated 
pneumonia and osteomyelitis) in 
which prolonged IV antibiotics were 
previously considered the standard of 
care.°”°° Prolonged courses of IV 


antibiotics administered via a PICC 
were associated with increased risk of 
catheter-associated complications 
and increased reutilization in these 
studies. In children who were able to 
tolerate oral antibiotics, parenteral 
antibiotic therapy did not confer a clear 
benefit over oral alternatives with 
excellent bioavailability.’ However, 
there are limited data on the 
bioavailability of oral antibiotics among 
infants <60 days old. High-dose oral 
amoxicillin (200 mg/kg per day) has 
been shown to achieve therapeutic 
serum concentrations in term neonates 
with S agalactiae bacteremia.** Further 
investigation is needed to establish 
optimal dosing and bioavailability for 
oral antibiotics typically used to treat 
urinary pathogens (eg, aminopenicillins 
and cephalosporins) in infants <60 
days old. 


Additionally, it is possible that there is 
a subset of young infants with 
bacteremic UTI in whom longer 
courses of parenteral antibiotics are 
indicated. For instance, 4 of the 6 
infants with recurrent UTI in our 
cohort were also diagnosed with VUR, 
and 3 of these infants had Grade 3 or 
4 VUR. High-grade VUR is associated 
with an increased risk of UTI 
recurrence.*”*° However, it is not 
known if longer duration of 
parenteral antibiotic therapy or 
longer duration of total therapy is 
associated with a reduced risk of 
recurrent UTI in infants with VUR. 
Future studies should focus on 
identifying which populations of 
young infants, if any, may benefit 
from longer courses of parenteral 
antibiotic therapy. 


CONCLUSIONS 


Young infants with bacteremic UTI 
who received <7 days of parenteral 
antibiotic therapy did not have 
significantly more 30-day recurrent 
UTIs or hospital reutilization than 
infants who received longer courses 
of parenteral therapy. Our data 
indicates that <7 days of parenteral 


antibiotic therapy may be safe in this 
population. Researchers in future 
prospective studies should seek to 
establish the bioavailability and 
optimal dosing of oral antibiotics in 
young infants and assess if there are 
particular subpopulations of infants 
with bacteremic UTI who may benefit 
from longer courses of parenteral 
antibiotic therapy. 
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ABBREVIATIONS 


CFU: colony-forming unit 

CI: confidence interval 

CSF: cerebrospinal fluid 

ED: emergency department 

IPW: inverse probability weighting 
IV: intravenous 


IV/IM: intravenous and/or 
intramuscular 
LOS: length of stay 
PICC: peripherally inserted central 
catheter 
UTI: urinary tract infection 
VUR: vesicoureteral reflux 
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